Instructor-Led | Virtual or In-Person | 1 Full or 2 Half-Days | 25 Participants → AI & ML

Testing AI and Machine Learning (001)


Description

Although there are several controversies and misunderstandings surrounding AI and machine learning, one thing is apparent — people have quality concerns about the safety, reliability, and trustworthiness of these types of systems. Not only are ML-based systems shrouded in mystery due to their largely black-box nature, they also tend to be unpredictable since they can adapt and learn new things at runtime. Validating ML systems is challenging and requires a cross-section of knowledge, skills, and experience from areas such as mathematics, data science, software engineering, cyber-security, and operations. In this course you'll be introduced to quality engineering (QE) for AI and machine learning. You’ll learn the fundamentals of AI and ML, including how intelligent agents are modeled, trained and developed. You'll then dive into approaches for validating ML models offline, prior to release, and online, continuously, post-deployment. By developing and executing a test plan for a live ML-based recommendation system, you'll experience the practical issues around testing AI first-hand.

Content
  • Course Introduction
  • Introductions
  • Course Goals
  • Foundations
  • Quality
  • Software Testing
  • AI and Machine Learning (ML)
  • QE Practices for AI/ML
  • Hands-On: Testing Supervised ML
  • Model Training
  • Practical: Classifier Performance
  • Practical: Regression Performance
  • Model Validation
  • Practical: K-Fold Cross-Validation
  • Hands-On: Testing AI for Fairness
  • What is Fairness in AI/ML?
  • Types of Fairness Risks
  • Practical: Fairness in Training Data
  • Practical: AI Fairness Toolkits
  • Hands-On: Testing Unsupervised ML
  • Cluster Validation Approaches
  • Practical: Silhouette Coefficient
  • Hands-On: Testing Adaptive AI/ML
  • Practical: Testing Adaptive Recommendations
  • Key Considerations
  • Hands-On: ML Operations
  • MLOps Lifecycle and Activities
  • Practical: MLOps with Google AI Platform
  • Course Wrap Up
  • Lessons Learned
  • Future Directions
Completion rules
  • All units must be completed