Library used in this project
The open-source recommendation library used in this work. This link points to the upstream library.
Recommenders library on GitHubRecommendation Pipelines at Scale with Databricks & Spark
Production data and ML pipelines that turn customer interactions into personalized product recommendations.
SYSTEM OVERVIEW
the pieces, connected.
A public sketch of the documented components and workflow. Internal interfaces are omitted.
Download full-resolution SVGTHE PROBLEM
Where the work started.
Merchants need product recommendations informed by customer behavior. Interaction data must move through reliable processing and ML pipelines before it can support personalized feeds.
MY CONTRIBUTION
What I built.
- Developed a production recommendation system on Databricks.
- Built pipelines to process customer interaction data for personalized product feeds.
- Worked across Python, Spark and ML workflows to connect data processing with recommendation delivery.
1M+user interactions processed daily
Delivered personalized product recommendations through production data and ML pipelines.
Selected work from my engineering role at SWYM. Impact figures are reported from this work; the companion examples below use synthetic data.
Inspect the work behind the explanation.
Ranking quality for recommended products
A new, runnable example that scores saved predictions against expected outputs. It uses fictional data and illustrates evaluation mechanics; it does not reproduce the production system or substantiate the impact figures above.
python3 evaluate_outputs.py recommendations.json report.jsonPython 3.10+ · Standard library only · Includes fixtures, metric definitions and tests