Lecture 3.0.18: CICD & Cloud Cost Estimation Github Action
Why It Matters
Embedding CI/CD and cost estimation transforms student data projects into reliable, reproducible research tools, protecting both scientific integrity and limited grant resources.
Key Takeaways
- •Treat data projects like medical devices: scaffold, CI/CD, cost budgeting.
- •Use a standardized folder scaffold for reproducibility and data integrity.
- •GitHub Actions automate safety checks, blocking faulty data commits.
- •Cloud cost components: compute, storage, data transfer; choose appropriate instance.
- •Estimate expenses with AWS pricing calculator before deploying capstone projects.
Summary
The lecture reframes data‑science projects as medical devices, emphasizing three pillars: a reproducible scaffold, continuous integration/deployment (CI/CD) for safety, and a realistic cloud‑cost budget. By mirroring device housing, safety circuits, and manufacturing budgets, students are urged to treat codebases with the same rigor as clinical tools.
A prescribed folder hierarchy—data (read‑only), source, notebooks, and an environment.yml checklist—ensures anyone can reproduce the work. GitHub Actions serve as the “robot” that triggers on each push, runs validation scripts (e.g., heart‑rate sanity checks), and blocks merges when tests fail, providing immediate red‑cross alerts.
The speaker illustrates the approach with a dummy patient CSV containing an impossible heart‑rate of 450. The CI pipeline catches the error, forces correction, and then displays a green check. He also likens cloud spending to renting an operating theater: compute, storage, and data transfer each incur costs, and choosing a “sedan” CPU over a “Ferrari” GPU can save thousands.
For capstone teams, the takeaway is clear: adopt the scaffold for reproducibility, embed CI/CD for automated QA, and run an AWS pricing calculator to verify feasibility. Projects that meet these three criteria are more trustworthy, less error‑prone, and financially sustainable.
Comments
Want to join the conversation?
Loading comments...