How to Build, Train and Deploy Your Own Recommender System â Part 2
We build a recommender system from the ground up with matrix factorization for implicit feedback systems. We then deploy the model to production in AWS.
Today we will be investigating how we can bring a traditional web application to the present/future by making it work with Docker containers. We will be using a typical Umbraco MVC app, which thousands of websites are built with, and so many developers love. I know itâs not the latest and sexiest technology out there, however wait until you see it with Docker.
There may be plans to migrate Umbraco to .NET Core (host in Linux, yay), but until then we are stuck with Windows and IIS. Without further delay, letâs get the ball rolling!
After itâs release in 2013, the popularity of Docker has grown through the years. It allowed one to run software in containers - a virtualization technology which is comparable to virtual machines in its practical use. However, unlike virtual machines, it is much more compact, and more dynamic in its resource usage where you do not have to permanently set aside resources for it. It enables portability where one can define images which are self-contained and isolated, with all the dependencies it needs.
These Docker images are portable in that we can distribute them for others to run and get exactly the same behavior everywhere, without the need to manually install all the elements that make up the image. The time savings is clear when you donât have to wait through all the number of installations. And you get the exact same running container as what the image creator had intended.
There are many uses of Docker, but for this post, we will focus on it being a perfect candidate to use in a development team, to help in development environment standardization. In addition to that, when you work in a team, there is nothing more frustrating than trying to fix a bug and in the end find out that it was caused by a slightly different configuration between developers. Using containers will make sure that all developers are working on the same environment. And because we are using containers, we can easily setup a new developer, and get them up and contributing quickly.
Umbraco is a .Net based CMS (Content Management System) and started around the year 2000 and even though it has had multiple versions ever since, its architecture has not changed much. Umbraco has proved its longevity in the industry, and remains as one of the most popular .NET based CMS for developers. So in comparison to Docker, an Umbraco-based website is a traditional application. But that cannot stop to take advantage of the benefits of Docker and combine it with the utility of Umbraco.
Umbraco is a .NET application (not .NET Core) first and foremost, and that is our main requirement. Docker was designed for Linux, but because it is not .NET Core, we cannot use Linux containers. However Docker now has support for Windows containers so we can use this with Umbraco. Their support for Windows is getting better although they are still not at par with the capabilities of their Linux cousins.
We will be using Docker Compose to define our system, which will have 2 parts (or services as per docker-compose lingo). The first service is the Umbraco DB - an SQL based database which is the center of an Umbraco project. The second service is the .NET based website that houses the Umbraco system. Both services are defined in a docker-compose.yml as weâll see in the next section. This defines our system, the portable image that we can pass around our development team.
This is an over simplified Docker system, a typical project might have multiple APIs referenced in the same docker compose file, with the idea that you can define your whole system in your computer, only limited by the your hardware.
Source code for the following exercise is found here.
version: '3.4'
services:
umbraco-db:
build:
context: ..\DevEnvironment\umbraco-db
container_name: umbraco-sqldb
volumes:
- C:\ProgramData\Docker\volumes\umbraco-volume:C:\volumes
ports:
- "1433:1433"
environment:
SA_PASSWORD: "StrongP@ssw0rd!"
ACCEPT_EULA: "Y"
# attach_dbs: "[{'dbName':'umbraco-cms','dbFiles':['C:\\\\volumes\\\\umbraco-cms.mdf','C:\\\\volumes\\\\umbraco-cms_log.ldf']}]"
image: microsoft/mssql-server-windows-developer
networks:
- default
umbracodocker:
image: ${DOCKER_REGISTRY-}umbracodocker
depends_on:
- umbraco-db
build:
context: .\UmbracoDocker
dockerfile: Dockerfile
After doing so, you can now run the web application by Ctrl-F5:
Tip #1 - To run the website, do a Ctrl-F5 (Start without debugging). Also make sure that you have the âdocker-composeâ project as the startup project. Doing so will run docker-compose on it, running all the containers defined in the compose file, and open a browser with Umbraco ready for configuration. There seems to be a problem with Visual Studio if you chose to debug it using F5 (Start Debugging).
Tip #2 - Running Windows-based containers require Hyper-V enabled on your PC through a BIOS setting. However in a Bootcamp partition on a Mac like mine, there is no BIOS setting. The workaround for this to first boot up into your Mac. Choose Apple menu > System Preferences, then click Startup Disk. Click the lock icon and enter your administrator password. Select your Windows partition as the startup disk, then restart by clicking the âRestartâ button next to it. This is important I think, as it enables the hardware virtualization mode, even after you reboot to the Windows partition. What a hack.
Tip #3 - To find the IP address of a Docker container, type
docker inspect <containerId>
. The containerâs IP address will be displayed towards the bottom of the output.
docker-compose down
you will retain the data written to by the running Docker container.Tip #4 - You will have to manually update UmbracoDBSN to use the IP address of umbraco-db since there is an existing bug in windows containers wherein you cannot use localhost. Also if you want to load an existing *.mdf and *.ldf, you have to attach them in the docker-compose.yml using the attach_dbs environment variable. I have commented it in the docker-compose file as a guide to you. And one more thing, you have to add the key to appSettings:
<add key="Umbraco.Core.ConfigurationStatus" value="8.0.1" />
as Umbraco uses that key to mean that an installation has succeeded, and just load the DB in the mdf file. Otherwise, Umbraco will think that it is not yet installed and will proceed to the installation screens and that is not what we want.
Note that disk based artifacts like media files, are not covered in this, and these may be covered by another article, and is beyond the scope of this post
in addition to using Docker as a mechanism fo standardizing our development environment, why donât you go one step further. You can utilize Docker fully by leveraging on itâs deployment capabilities, that is, deployment to your staging and production environments. Container orchestration software such as Kubernetes, Docker Swarm or Terraform are options that can be explored.
If you have reached this part, you have a patience of a monk. In this rather long article, we have explored the viability of using Docker containers as a strategy for making local dev environments easy to work with.
When a new developer starts, it will be a breeze to just use the images and get them up and productive in no time. But it doesnât have to be just Umbraco projects. Docker containers can be used for many different types and classes of applications, old and new. Maybe in a later post we can continue where we left off and try our hand with orchestration software to extend our use of Docker for deployments too.
Until then, see ya.
We build a recommender system from the ground up with matrix factorization for implicit feedback systems. We then deploy the model to production in AWS.
We build a recommender system from the ground up with matrix factorization for implicit feedback systems. We put it all together with Metaflow and used Comet...
Building and maintaining a recommender system that is tuned to your businessâ products or services can take great effort. The good news is that AWS can do th...
Provided in 6 weekly installments, we will cover current and relevant topics relating to ethics in data
Get your ML application to production quicker with Amazon Rekognition and AWS Amplify
(Re)Learning how to create conceptual models when building software
A scalable (and cost-effective) strategy to transition your Machine Learning project from prototype to production
An Approach to Effective and Scalable MLOps when youâre not a Giant like Google
Day 2 summary - AI/ML edition
Day 1 summary - AI/ML edition
What is Module Federation and why itâs perfect for building your Micro-frontend project
What you always wanted to know about Monorepos but were too afraid to ask
Using Github Actions as a practical (and Free*) MLOps Workflow tool for your Data Pipeline. This completes the Data Science Bootcamp Series
Final week of the General Assembly Data Science bootcamp, and the Capstone Project has been completed!
Fifth and Sixth week, and we are now working with Machine Learning algorithms and a Capstone Project update
Fourth week into the GA Data Science bootcamp, and we find out why we have to do data visualizations at all
On the third week of the GA Data Science bootcamp, we explore ideas for the Capstone Project
We explore Exploratory Data Analysis in Pandas and start thinking about the course Capstone Project
Follow along as I go through General Assemblyâs 10-week Data Science Bootcamp
Updating Context will re-render context consumers, only in this example, it doesnât
Static Site Generation, Server Side Render or Client Side Render, whatâs the difference?
How to ace your Core Web Vitals without breaking the bank, hint, its FREE! With Netlify, Github and GatsbyJS.
Follow along as I implement DynamoDB Single-Table Design - find out the tools and methods I use to make the process easier, and finally the light-bulb moment...
Use DynamoDB as it was intended, now!
A GraphQL web client in ReactJS and Apollo
From source to cloud using Serverless and Github Actions
How GraphQL promotes thoughtful software development practices
Why you might not need external state management libraries anymore
My thoughts on the AWS Certified Developer - Associate Exam, is it worth the effort?
Running Lighthouse on this blog to identify opportunities for improvement
Use the power of influence to move people even without a title
Real world case studies on effects of improving website performance
Speeding up your site is easy if you know what to focus on. Follow along as I explore the performance optimization maze, and find 3 awesome tips inside (plus...
Tools for identifying performance gaps and formulating your performance budget
Why web performance matters and what that means to your bottom line
How to easily clear your Redis cache remotely from a Windows machine with Powershell
Trials with Docker and Umbraco for building a portable development environment, plus find 4 handy tips inside!
How to create a low cost, highly available CDN solution for your image handling needs in no time at all.
What is the BFF pattern and why you need it.