Democratizing Electrochemical Analysis with Vision-Languageg AI - From 5% to 95% Accuracy

Timeline: 10 weeks

Role: Applied ML Intern

Team: Fischell Institute of Biomedical Devices

🎯 Project Overview

What: Fine-tuned Qwen2.5-VL to analyze electrochemical graphs (CVs/DPVs), extracting quantitative features and providing scientific interpretations for integration into autonomous laboratory systems.
Why: Electrochemistry offers unparalleled speed and accuracy for chemical measurements, making it highly desirable across research and industry. However, the specialized and nuanced analysis required to interpret the data creates a major bottleneck. This model solves that barrier, enabling both human researchers and agentic systems to leverage electrochemistry’s full potential.
Impact: Achieved 95% accuracy on peak detection and electrochemical question-answering, creating a robust tool for autonomous experimental workflows and data interpretation pipelines.

Quantified Performance

🔧 Technical Implementation

Key Technologies & Tools

Model & Framework: Fine-tuned multimodal Qwen2.5-VL using ms-swift framework
Training Techniques: Supervised Fine-Tuning (SFT), LoRA for compute efficiency, Full ViT tuning for visual understanding
Infrastructure: Remote hardware with >600 TFLOPS compute capacity
Data Pipeline: Python scripts and web tools using templating, augmentation, and dataset mixing strategies

Technical Approach

Developed a vision-language model that bridges electrochemical graph analysis with scientific reasoning. The model analyzes graphs for peak detection, curve comparison, anomaly detection, and concentration trends - processing visual features through fine-tuned encoders while maintaining chemical knowledge through language components.

Built comprehensive infrastructure including:

Interactive web apps for data labeling and quality verification
Structured output formatting for seamless integration with agentic systems
Evaluation suite for error analysis, failure mode tracking, and response visualization
APIs designed for autonomous lab agents, enabling closed-loop experimental workflows

Evaluation Tool

🚧 Challenges & Problem-Solving

Challenge: Limited raw electrochemical data

Solution: Built sophisticated data pipeline combining vision-language templating, automated augmentation, and synthetic generation. Developed web applications for efficient expert annotation and quality control. Scaled from limited raw data to 25,000 high-quality training samples through strategic augmentation while maintaining scientific validity.
Skills Demonstrated: Data engineering, web development, quality assurance systems

Challenge: Ensuring reliable integration with autonomous systems

Solution: Created structured output formats with confidence scores and error bounds. Implemented response parsing that extracts numerical values, peak coordinates, and interpretations in machine-readable formats for downstream agents.
Skills Demonstrated: API design, system integration, reliability engineering

Challenge: Understanding and improving model failures

Solution: Built comprehensive evaluation suite that visualizes model responses, tracks failure modes across different electrochemical scenarios, and identifies systematic errors. Enables rapid iteration by pinpointing exactly where and why the model struggles.
Skills Demonstrated: Error analysis, visualization tools, debugging complex systems

📊 Results & Impact

Graph Analysis Capabilities: Accurately extracts peaks, trends, and anomalies from electrochemical graphs
Integration Success: Deployed as analysis module in agentic experimental planning system
Performance Metrics: 95% accuracy with detailed error tracking across multiple electrochemical tasks
Evaluation Suite Benefits:
- Visualizes model predictions overlaid on original graphs
- Tracks failure modes (e.g., missing small peaks, baseline drift confusion)
- Enables targeted improvements through error pattern identification
Accessibility Impact: Non-electrochemists can now query graphs naturally and receive expert-level interpretations

What’s Next

Expanding integration with autonomous lab systems for closed-loop experimentation. The evaluation suite continues to guide improvements, particularly for edge cases in complex multi-electron processes.

💡 Key Takeaways

Technical Skills Developed:

Vision-language model training for scientific graph analysis
Data pipeline engineering: templating, augmentation, quality control
Web application development for scientific data annotation
Evaluation suite design with visualization and error tracking
API development for agentic system integration

Data Pipeline Innovations:

Templating System: Combined vision-language templates that preserve scientific relationships
Augmentation Strategy: Domain-aware augmentation maintaining electrochemical validity
Quality Control: Web-based tools for expert verification and iterative refinement
Scaling Methods: Grew dataset from limited raw data to 25k samples without quality loss

Evaluation Framework:

Error Visualization: Overlay predictions on original graphs for intuitive debugging
Failure Mode Tracking: Systematic categorization of error types
Performance Monitoring: Skill-specific metrics with confusion matrices
Iterative Improvement: Used insights from evaluation suite to achieve 10 weeks of consistent gains

🔗 Links

Research Lab: [Lab Website]

Electrochemical Multimodal LLM