Machine Learning Lifecycle for MLOps era brings model and software development together to build ML-assisted products — Building machine learning products or ML-assisted product features involve two distinct disciplines: Model Development: Data Scientists — highly skilled in statistics, linear algebra, and calculus — train, evaluate, and select the best-performing statistical or neural network model.

Scalable and efficient data pipelines are as important for the success of analytics, data science, and machine learning as reliable supply lines are for winning a war. — Updated paywall-free version: Scalable Efficient Big Data Pipeline Architecture For deploying big-data analytics, data science, and machine learning (ML) applications in the real world, analytics-tuning and model-training is only around 25% of the work. Approximately 50% of the effort goes into making data ready for analytics and ML. The remaining…

Difference between SQL and NoSQL databases. Deep dive, decision tree, and cheatsheet to choose the best for your data type and use case from 12 database types. — Paywall-free version: SQL vs. NoSQL How do you choose a database? Maybe, you assess whether the use case needs a Relational database. Depending on the answer, you pick your favorite SQL or NoSQL datastore, and make it work. …

Evolution of ML lifecycle from resource-constrained batch data mining to MLOps at the cloud scale — Everyone has been talking about MLOps for over a year now. I looked around for how the lifecycle and processes have evolved. The discipline of seeking insight from data has been around for 25 years. Back then, it was known as data mining. In this article, I present a survey…

Understand the cloud deployment landscape and pick the best alternative for your app, team, and business. — Amazon launched its first AWS services in 2006: S3 for Storage and EC2 for Compute. Now there are a plethora of services and offerings from various cloud vendors like AWS, Azure, and Google Cloud. It can be overwhelming to evaluate so many options and pick the right one for you. …

Various types of SQL Joins and how to make them fast — For data engineers and data scientists, SQL is valuable expertise. All modern data warehouses as well as data or delta lakes have SQL interfaces. Understanding how an SQL query is executed can help you write fast and economical queries. Once you understand SQL concepts like join and window, you will…

Best places to find free datasets to kickstart your machine learning and data science projects — Curated public datasets are widely used to learn data science and machine learning. But their utility in real-world commercial projects is often overlooked. You must have heard that data scientists spend 80% of their time collecting, cleaning, and preparing the data. Evaluating the viability of an idea will require a…

What if programming languages were stocks? And you had to make a portfolio to fetch good returns in 2023? — You probably have seen various surveys and analyses listing the most popular programming languages. Those cover the universe of programming languages, like the S&P 500 does for the stock market. What would be the best portfolio for you to be successful and outperform the rest? Of course, it depends on…

Systematic way of collecting data and extracting actionable insights using the Drivetrain Approach. — “Let’s collect all data we can, and we will fish for insights later.” Have you heard this before? That approach seldom works. On rare occasions when it does work a little, the RoI is very low w.r.t. the cost of collecting, processing, and storing volumes of data. …