How to Build, Train and Deploy Your Own Recommender System â Part 2
We build a recommender system from the ground up with matrix factorization for implicit feedback systems. We then deploy the model to production in AWS.
This is my Day 2 report for the AWS Summit 2022 AU and NZ.
I make no secret that my goal in the next 12 months is to build up Machine Learning skills and find projects that will exploit these newfound skills. So many of my learning activities this past year have been about this field, and the effort is still continuing. Imagine my surprise when I found out that the AWS Summit has a good chunk of sessions focusing solely on AI/ML!
In the Day 2 Keynote, Simon Elisha - ANZ Chief Technologist, was talking excitedly about a few cloud advancements but it was these AWS Partnerâs presentations that attracted my attention:
Franky is an at-home rehabilitation robot for stroke patients that uses AWS IOT and Machine Learning among other technologies.
In early trials, Franky is showing the benefits of using AWS IoT by saving patients travelling time, improving data collection, and avoiding physical contact with patients during the pandemic.
Swoop Aero is one cool company, using cool technology. It has been used all around the world to as a world-leading drone logistics service, delivering anywhere on the planet, all from there operation HQ in Melbourne!
All made possible by proprietary drone aircraft - Kite, AI technology and digital twin based on AWS.
The damage and loss caused by bushfires every year is in the excess of tens of millions of hectares and would have cost billions of dollars to the economy. It has been identified that one of leading cause of bushfires originate from the energy supply network, ie. the millions of kilometers of power lines all around the country.
To improve the process of asset inspection of these power poles nationwide, Machine Learning was used to replace the current method to ensure that the operation gets completed quicker and cheaper. The current method was very labor intensive and canât possibly scale easily nationwide.
Through a project collaboration between the NSW Government and University of Sydney a project was undertaken to tackle the problem of diagnosing Sepsis in the emergency room. Sepsis is a leading cause of death in Australia, greater than the annual national road toll, breast cancer, prostate cancer, or colon cancer.
However if it can be diagnosed early enough, then most of these deaths will have been avoided. This session demonstrated the use of model ensembles (in this case 3 models - XGBoost, Linear Learner and Logistic Regression), and AWS SageMaker, Amazonâs machine learning platform.
This next session was quite interesting as it combines IOT, Edge, reinforcement learning and computer vision all in the same project, aiming to replace the current decades old technology being used in our roads.
There is a need to optimize the current state of things because our cityâs livability has now been affected. Pollution, Noise, accidents, and congestion, there has to be something that can be done with these.
AWS IoT Greengrass allows one to manage their IOT Edge devices, download ML models locally, so that inference can then be also be done locally.
The last (and definitely not the least), is the talk from Carsales on how they scaled their Data Science/ML capability. As they donât have a large DS team but their projects that required Data Science were on the rise, they have demonstrated how they were able to:
As a result of their new DS framework (based on a Metaflow - a DS framework built at Netflix and AWS SageMaker Pipelines), they were able to free up their DS resources so that Software Developers were now trained and equipped to tackle their normal DS projects, at a ratio of 70% DS/ML work was now completed by developers. This leaves the 30% meatier and more difficult problems for the Data Scientists to tackle.
A couple of highlights for today, first was the demo from the team at Swoop Aero, and to think that they do all their world operations from their Melbourne HQ is just mind boggling.
And next is Carsales as they were able to democratize the Data Science and Machine Learning, they magically enabled their developer team to tackle Data Science and Machine Learning projects. They developed a Data Science framework that enabled normal developers to work on DS/ML projects with little input from the DS team, which frees up the Data Scientists to work on the more difficult problems facing the business.
We build a recommender system from the ground up with matrix factorization for implicit feedback systems. We then deploy the model to production in AWS.
We build a recommender system from the ground up with matrix factorization for implicit feedback systems. We put it all together with Metaflow and used Comet...
Building and maintaining a recommender system that is tuned to your businessâ products or services can take great effort. The good news is that AWS can do th...
Provided in 6 weekly installments, we will cover current and relevant topics relating to ethics in data
Get your ML application to production quicker with Amazon Rekognition and AWS Amplify
(Re)Learning how to create conceptual models when building software
A scalable (and cost-effective) strategy to transition your Machine Learning project from prototype to production
An Approach to Effective and Scalable MLOps when youâre not a Giant like Google
Day 2 summary - AI/ML edition
Day 1 summary - AI/ML edition
What is Module Federation and why itâs perfect for building your Micro-frontend project
What you always wanted to know about Monorepos but were too afraid to ask
Using Github Actions as a practical (and Free*) MLOps Workflow tool for your Data Pipeline. This completes the Data Science Bootcamp Series
Final week of the General Assembly Data Science bootcamp, and the Capstone Project has been completed!
Fifth and Sixth week, and we are now working with Machine Learning algorithms and a Capstone Project update
Fourth week into the GA Data Science bootcamp, and we find out why we have to do data visualizations at all
On the third week of the GA Data Science bootcamp, we explore ideas for the Capstone Project
We explore Exploratory Data Analysis in Pandas and start thinking about the course Capstone Project
Follow along as I go through General Assemblyâs 10-week Data Science Bootcamp
Updating Context will re-render context consumers, only in this example, it doesnât
Static Site Generation, Server Side Render or Client Side Render, whatâs the difference?
How to ace your Core Web Vitals without breaking the bank, hint, its FREE! With Netlify, Github and GatsbyJS.
Follow along as I implement DynamoDB Single-Table Design - find out the tools and methods I use to make the process easier, and finally the light-bulb moment...
Use DynamoDB as it was intended, now!
A GraphQL web client in ReactJS and Apollo
From source to cloud using Serverless and Github Actions
How GraphQL promotes thoughtful software development practices
Why you might not need external state management libraries anymore
My thoughts on the AWS Certified Developer - Associate Exam, is it worth the effort?
Running Lighthouse on this blog to identify opportunities for improvement
Use the power of influence to move people even without a title
Real world case studies on effects of improving website performance
Speeding up your site is easy if you know what to focus on. Follow along as I explore the performance optimization maze, and find 3 awesome tips inside (plus...
Tools for identifying performance gaps and formulating your performance budget
Why web performance matters and what that means to your bottom line
How to easily clear your Redis cache remotely from a Windows machine with Powershell
Trials with Docker and Umbraco for building a portable development environment, plus find 4 handy tips inside!
How to create a low cost, highly available CDN solution for your image handling needs in no time at all.
What is the BFF pattern and why you need it.