How to Build, Train and Deploy Your Own Recommender System – Part 2
We build a recommender system from the ground up with matrix factorization for implicit feedback systems. We then deploy the model to production in AWS.
The next micro-frontends building block we will cover is Module Federation.
It was initially created to enable asynchronous loading of Javascript bundles, so one can easily share and consume code in the Javascript ecosystem, ie. any Javascript project, think browser, Node.js or Electron.
Later on, it was extended to support server-side rendering scenarios. Since the method I’m going to explore in building my micro-frontends is client-side composition using React.js, I’ve identified Module Federation as the perfect technology to use.
Module Federation was created by Zack Jackson, a Webpack core maintainer himself, and was integrated as a flagship feature of Webpack 5 in October 2020.
Scalability - its the problem that Module Federation tries to solve.
So that large applications can be split easily where each part can then be shared among other parts and may be developed by separate teams.
Module Federation is a game-changer in Javascript Architecture because before this, sharing code was clunky and just didn’t feel smooth enough. The usual module sharing method that we have been using for years is using NPM packages, enabling developers to create build-time dependencies. Although NPM works, it cannot help us when we need to load dependencies at run-time.
As micro-frontends evolved, we now have a need to resolve Javascript modules at run-time. This is where Module Federation shines.
Because Module Federation encourages you to arrange your application into separate projects so that you can build and deploy them independently (therefore in parallel), each project can be built and deployed in isolation and may be done by different teams.
The problem with the usual NPM module composition is that it typically increases the application size as the number of dependencies increase. Module Federation provides you an option to lazy load bundles, to avoid loading them when your application loads, but only loading them on demand. This results in a better web performance as it avoids having to download modules before they are actually needed.
And because Module Federation provides excellent dependency management, it efficiently resolves vendor and third-party dependencies so that only one version of a library is ever loaded by your application. How good is that?
Instead of sharing code and thinking of “libraries” when using NPM package approach, we can think of applications that use Module Federation not dissimilar to APIs. Now web applications can expose functionality to other applications, the same way that it can also consume from other applications.
The ability to have evergreen functionality is very appealing to the developer. There will be no need to redeploy the consumers anymore when exposed dependent functionality will have changed. I have to say that this in itself is a very powerful feature, which will need very careful consideration to avoid unintended results.
With shared dependencies, Module Federation maintains a dependency graph for your whole application, so that even though applications fail to declare a dependency or when there are network issues, it knows the needed dependencies so that it takes care of downloading it as required.
Bringing in shared functionality to your application is quite simple - either synchronous loading by importing the bundle as usual. Or asynchronous loading by using lazy loading, to only load the dependencies when required.
Using Module Federation will feel very familiar to any Javascript developer, since it is available as a Webpack plugin starting with Webpack 5. If we think about that for a bit, this is actually quite powerful and exciting.
Think about all the things that Webpack can bundle - scripts, assets, styles, images, markdowns, and more by using third party Webpack loaders. All these can be federated and shared though Module Federation, great, right?
However, as great as Module Federation is, it is important to remember that it is not a framework, and as such it does not handle any of the implementation details for you. For example, you could use awesome client-side composition libraries such as Single SPA, or Next.js and just leverage Module Federation to do the module loading over the wire for you.
In this article we have introduced Module Federation as an excellent option for building your micro-frontend application.
It promotes scalability by enabling multiple teams to work on a separate applications by allowing applications to share and consume functionality at runtime.
It makes applications more compact through the use of shareable dependencies.
It enables evergreen functionality so that you won’t need to build and deploy your consumers when the shared functionality changes.
Developer experience is great since many developers are already familiar with the Webpack ecosystem
Once configured, your application will work like a monolith which is awesome.
We build a recommender system from the ground up with matrix factorization for implicit feedback systems. We then deploy the model to production in AWS.
We build a recommender system from the ground up with matrix factorization for implicit feedback systems. We put it all together with Metaflow and used Comet...
Building and maintaining a recommender system that is tuned to your business’ products or services can take great effort. The good news is that AWS can do th...
Provided in 6 weekly installments, we will cover current and relevant topics relating to ethics in data
Get your ML application to production quicker with Amazon Rekognition and AWS Amplify
(Re)Learning how to create conceptual models when building software
A scalable (and cost-effective) strategy to transition your Machine Learning project from prototype to production
An Approach to Effective and Scalable MLOps when you’re not a Giant like Google
Day 2 summary - AI/ML edition
Day 1 summary - AI/ML edition
What is Module Federation and why it’s perfect for building your Micro-frontend project
What you always wanted to know about Monorepos but were too afraid to ask
Using Github Actions as a practical (and Free*) MLOps Workflow tool for your Data Pipeline. This completes the Data Science Bootcamp Series
Final week of the General Assembly Data Science bootcamp, and the Capstone Project has been completed!
Fifth and Sixth week, and we are now working with Machine Learning algorithms and a Capstone Project update
Fourth week into the GA Data Science bootcamp, and we find out why we have to do data visualizations at all
On the third week of the GA Data Science bootcamp, we explore ideas for the Capstone Project
We explore Exploratory Data Analysis in Pandas and start thinking about the course Capstone Project
Follow along as I go through General Assembly’s 10-week Data Science Bootcamp
Updating Context will re-render context consumers, only in this example, it doesn’t
Static Site Generation, Server Side Render or Client Side Render, what’s the difference?
How to ace your Core Web Vitals without breaking the bank, hint, its FREE! With Netlify, Github and GatsbyJS.
Follow along as I implement DynamoDB Single-Table Design - find out the tools and methods I use to make the process easier, and finally the light-bulb moment...
Use DynamoDB as it was intended, now!
A GraphQL web client in ReactJS and Apollo
From source to cloud using Serverless and Github Actions
How GraphQL promotes thoughtful software development practices
Why you might not need external state management libraries anymore
My thoughts on the AWS Certified Developer - Associate Exam, is it worth the effort?
Running Lighthouse on this blog to identify opportunities for improvement
Use the power of influence to move people even without a title
Real world case studies on effects of improving website performance
Speeding up your site is easy if you know what to focus on. Follow along as I explore the performance optimization maze, and find 3 awesome tips inside (plus...
Tools for identifying performance gaps and formulating your performance budget
Why web performance matters and what that means to your bottom line
How to easily clear your Redis cache remotely from a Windows machine with Powershell
Trials with Docker and Umbraco for building a portable development environment, plus find 4 handy tips inside!
How to create a low cost, highly available CDN solution for your image handling needs in no time at all.
What is the BFF pattern and why you need it.