Scalable and efficient data pipelines are as important for the success of analytics, data science, and machine learning as reliable supply lines are for winning a war. — For deploying big-data analytics, data science, and machine learning (ML) applications in the real world, analytics-tuning and model-training is only around 25% of the work. Approximately 50% of the effort goes into making data ready for analytics and ML. The remaining 25% effort goes…

Difference between SQL and NoSQL databases. Deep dive, decision tree, and cheatsheet to choose the best for your data type and use case from 12 database types. — How do you choose a database? Maybe, you assess whether the use case needs a Relational database. Depending on the answer, you pick your favorite SQL or NoSQL datastore, and make it work. It is a prudent tactic: a known devil is better than an unknown angel.

You define logic in traditional programs. Machine Learning extracts logic (models) from the data. — In the previous issue, we examined the MLOps ecosystem. It is a lot more complex compared to traditional software engineering projects. In this issue, let’s understand the differences between: Traditional programs and Machine Learning Software Development Life Cycle and ML Project Life Cycle We will also examine the evolution of…

Understand the cloud deployment landscape and pick the best alternative for your app, team, and business. — Amazon launched its first AWS services in 2006: S3 for Storage and EC2 for Compute. Now there are a plethora of services and offerings from various cloud vendors like AWS, Azure, and Google Cloud. It can be overwhelming to evaluate so many options and pick the right one for you. …

Various types of SQL Joins and how to make them fast — For data engineers and data scientists, SQL is valuable expertise. All modern data warehouses as well as data or delta lakes have SQL interfaces. Understanding how an SQL query is executed can help you write fast and economical queries. Once you understand SQL concepts like join and window, you will…

Best places to find free datasets to kickstart your machine learning and data science projects — Curated public datasets are widely used to learn data science and machine learning. But their utility in real-world commercial projects is often overlooked. You must have heard that data scientists spend 80% of their time collecting, cleaning, and preparing the data. Evaluating the viability of an idea will require a…