LLM/GenAI Project Requirements
Phase 1. Framing the Problem
Before any modeling: what the problem is, why it matters, and how it becomes a machine learning task.
Requirement 1
Problem Domain Understanding
Demonstrate a thorough understanding of the problem domain, its motivation, and real-world significance. Justify why the problem matters using credible evidence such as research literature, industry reports, statistics, market data, or other factual sources.
Common anti-patterns
- Vague or generic problem statement.
- Motivation based on personal opinion rather than evidence.
- No quantitative justification.
- Poor understanding of existing solutions and their limitations.
- Selecting a trivial or artificial problem.
Requirement 2
Problem Significance and Value
Address a clearly defined, meaningful, and preferably unmet need. The project should solve a problem that delivers tangible value to its intended users or stakeholders, with well-defined objectives and success criteria.
Common anti-patterns
- Solving a trivial problem.
- Addressing a problem already well solved without differentiation.
- Vague objectives or unclear target users.
- No convincing explanation of expected impact.
- Feature-driven rather than problem-driven development.
Requirement 3
Problem Formulation
Translate the real-world need into a well-defined machine learning problem. Clearly specify the prediction or decision task, inputs, outputs, assumptions, constraints, and evaluation objectives.
Common anti-patterns
- Incorrect ML task formulation.
- Poorly defined target variable.
- Mismatch between the business problem and the ML formulation.
- Ignoring assumptions or constraints.
- Framing the problem around available data rather than the actual need.
Phase 2. Research, Data, and Baselines
What already exists, where the data comes from, and what the solution has to beat.
Requirement 4
Novelty
The problem you solve, or the way you solve it, must contain a significantly novel element. It should not already be solved, and it should differ meaningfully from how similar problems have been solved before. The novelty can lie in the problem itself, the data, the method, or the application, but it has to be real and identifiable.
Common anti-patterns
- Re-solving a problem that is already well solved.
- Reimplementing an existing project with only cosmetic changes.
- Claiming novelty without checking what already exists.
- A novel-sounding title over a standard, off-the-shelf solution.
- Confusing “new to me” with “new to the field”.
Requirement 5
State-of-the-Art Solution Research
Conduct a thorough review of relevant state-of-the-art models, methods, and technologies. Demonstrate a deep understanding of how these approaches work, their strengths and limitations, and explain their relevance to the project.
Common anti-patterns
- Choosing models because they are popular.
- Superficial descriptions.
- Comparing methods without understanding them.
- Ignoring recent literature.
- Weak justification for model selection.
Requirement 6
Data Acquisition and Generation Methodology
Develop and justify a sound methodology for acquiring, generating, or curating data using principles learned in the course. The methodology should be systematic, reproducible, and aligned with the project objectives.
Common anti-patterns
- Arbitrary dataset selection.
- Poor-quality or biased data.
- No documented acquisition methodology.
- Ignoring ethical or legal considerations.
- Data that poorly represents the target problem.
Requirement 7
Exploratory Data Analysis and Baseline Development
Perform thorough exploratory data analysis (EDA) to understand the dataset, identify quality issues, discover meaningful patterns, and guide modeling decisions. Establish strong baseline models for comparison.
Common anti-patterns
- Superficial EDA.
- Ignoring missing values, outliers, imbalance, or leakage.
- Skipping baseline models.
- Weak baselines.
- Decisions unsupported by data analysis.
Phase 3. Evaluation and Delivery
Comparing real alternatives, reporting them honestly, and shipping a repository someone else can run.
Requirement 8
Comparative Evaluation of Multiple Approaches
Implement and compare several meaningful, non-trivial modeling approaches using fair experimental methodology and appropriate evaluation metrics, and analyze the trade-offs between them. At least one approach must be a trained or fine-tuned model: the project cannot rely only on zero-shot or few-shot prompting.
Common anti-patterns
- Relying only on zero-shot or few-shot prompting, with no trained or fine-tuned model.
- Evaluating only one model.
- Comparing only minor parameter variations.
- Unfair comparisons.
- Reporting results without analysis.
- Choosing the best model solely by a single metric.
Requirement 9
Results Presentation and Communication
Present results clearly using appropriate evaluation metrics, informative visualizations, and concise technical explanations. Clearly communicate both performance and key insights.
Common anti-patterns
- Reporting metrics without interpretation.
- Poor evaluation metrics.
- Unclear or cluttered visualizations.
- Overloaded reports or presentations.
- Conclusions unsupported by evidence.
Requirement 10
Repository Quality and Reproducibility
Maintain a professional GitHub repository with a meaningful name, comprehensive README, logical organization, and all necessary artifacts to enable reproducibility. Submit the latest version of every artifact, and keep the code, data, README, and slides (PPT) aligned with each other, so the numbers, figures, and claims are consistent across all of them.
Common anti-patterns
- Poor repository organization.
- Missing README.
- Disorganized project structure.
- Missing artifacts required for reproduction.
- Lack of diagrams, screenshots, or workflow documentation.
- Submitting stale artifacts, or code, data, README, and slides that disagree on the results.
Phase 4. Rigor, Progress, and Depth
The difference between a project that ran once and a project that was actually engineered.
Requirement 11
Methodological Rigor and Iterative Improvement
Demonstrate substantial and disciplined effort through systematic experimentation, controlled comparisons, exploration of hyperparameters and design choices, data validation, error analysis, and iterative refinement. The methodology should be sound whether the project is primarily an engineering build or a scientific investigation.
Common anti-patterns
- Stopping after the first working model.
- Minimal hyperparameter tuning.
- No data validation.
- No error analysis.
- No iterative improvement cycle.
- Blind reliance on AutoML or AI-generated code.
Requirement 12
Project Progression and Responsiveness to Feedback
Demonstrate continuous progress throughout the semester by achieving meaningful milestones, incorporating instructor feedback, and refining the solution over multiple iterations. The final project should clearly reflect learning and technical growth.
Common anti-patterns
- Minimal progress after the proposal.
- Ignoring instructor feedback.
- Last-minute implementation.
- Repeating the same mistakes.
- No evidence of improvement between milestones.
Requirement 13
Technical Sophistication and Critical Thinking
Build a technically sophisticated solution, one that required many non-trivial design decisions, and be able to justify each of them. Go beyond obvious or common-sense choices by identifying subtle challenges, domain-specific nuances, hidden assumptions, edge cases, and trade-offs. Depth shows in the decisions you can defend, not in the size of the pipeline.
Common anti-patterns
- A trivial pipeline with few real design decisions.
- Superficial analysis.
- Treating models as black boxes.
- Ignoring data or domain nuances.
- Generic, textbook-level discussion.
- Repeating AI-generated explanations without critical evaluation.
- Failing to justify technical decisions or discuss trade-offs.
For Graduate Students. Additional Requirements
Beyond the thirteen above, a graduate project is held to a research standard: a real contribution, rigorous evaluation, and a paper you could submit.
Graduate 1
Substantial Problem or Application
Address a valuable real-world, scientific, or engineering problem. A toy example or small demonstration may be used for initial validation but is insufficient as the main project.
Graduate 2
Deep and Structured Review of Prior Work
Read relevant recent papers and systematically map existing methods, datasets, assumptions, evaluation approaches, strengths, and limitations. Clearly identify the research gap addressed by the project.
Graduate 3
Clear Research Contribution
Define a non-trivial contribution, such as a new method, meaningful extension, dataset, benchmark, evaluation framework, or rigorous empirical finding. Simply integrating existing components is insufficient.
Graduate 4
Multiple Strong Models, Including SOTA
Build and compare at least three genuinely different models, including both a fine-tuned model and zero-shot or few-shot (in-context) variants, alongside appropriate baselines and at least one recent state-of-the-art method. Explain and justify all major design choices.
Graduate 5
Research-Grade Experimental Evaluation
Conduct controlled experiments, including ablation studies, repeated runs, robustness testing, error analysis, and evaluation across multiple datasets, conditions, or relevant scenarios when feasible.
Graduate 6
Well-Written Academic Report
Produce a publication-quality mini-paper suitable, after refinement, for submission to arXiv. It should include an abstract, motivation, related work, methodology, experimental setup, results, discussion, limitations, and conclusions, with claims supported by evidence and properly cited sources.
