Scalable and efficient data pipelines are as important for the success of analytics, data science, and machine learning as reliable supply lines are for winning a war. — For deploying big-data analytics, data science, and machine learning (ML) applications in the real world, analytics-tuning and model-training is only around 25% of the work. Approximately 50% of the effort goes into making data ready for analytics and ML. The remaining 25% effort goes into making insights and model inferences…

Difference between SQL and NoSQL databases. Deep dive, decision tree, and cheatsheet to choose the best for your data type and use case from 12 database types. — How do you choose a database? Maybe, you assess whether the use case needs a Relational database. Depending on the answer, you pick your favorite SQL or NoSQL datastore, and make it work. It is a prudent tactic: a known devil is better…

Issue 13: Don’t fall for the “AI-first” hype. Focus on being “data-first” and ML/AI will naturally follow. — I learned about this question in a panel discussion at a conference last week: While most leaders acknowledge AI’s role in driving growth, companies are still figuring out how to change themselves to meet it. What is your advice on getting the scope of AI initiatives just right? …

Evolution of ML lifecycle from resource-constrained batch data mining to MLOps at the cloud scale — Everyone has been talking about MLOps for over a year now. I looked around for how the lifecycle and processes have evolved. The discipline of seeking insight from data has been around for 25 years. Back then, it was known as data mining. In this article, I present a survey…

Issue 12: The answer, as usual, is: it depends. — What is MLOps? MLOps stands for Machine Learning Operations. It is a set of practices to continuously train, integrate, and deploy (CT/CI/CD) Machine Learning models in production. It is inspired by DevOps practices followed in software development. MLOps is for managing the entire Machine Learning lifecycle — from formulating an ML problem from…

Issue 11: You define logic in traditional programs. Machine Learning extracts logic (models) from the data. — In the previous issue, we examined the MLOps ecosystem. It is a lot more complex compared to traditional software engineering projects. In this issue, let’s understand the differences between: Traditional programs and Machine Learning Software Development Life Cycle and ML Project Life Cycle

Issue 10: Overview of MLOps, Pipeline, and Maturity Levels — MLOps is a hot topic and everyone seems to be talking about it. I have been reading quite a bit of material, but I will share what I have learned so far. Why MLOps Since quite a lot of ML projects fail to go to production, MLOps is tooling, processes, and best…

Understand the cloud deployment landscape and pick the best alternative for your app, team, and business. — Amazon launched its first AWS services in 2006: S3 for Storage and EC2 for Compute. Now there are a plethora of services and offerings from various cloud vendors like AWS, Azure, and Google Cloud. It can be overwhelming to evaluate so many options and…

Issue 9: ML may not be the right solution to your problem. — In the previous issue, I discussed why Machine Learning projects fail. In this issue, let’s start figuring out how to build successful Machine Learning products. The first step is understanding when Machine Learning is more effective than traditional programming. Traditional programs have deterministic logic to solve a problem. But Machine…