{"id":3452,"date":"2025-04-30T07:02:23","date_gmt":"2025-04-30T07:02:23","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2025\/04\/30\/data-analyst-or-data-engineer-or-analytics-engineer-or-bi-engineer\/"},"modified":"2025-04-30T07:02:23","modified_gmt":"2025-04-30T07:02:23","slug":"data-analyst-or-data-engineer-or-analytics-engineer-or-bi-engineer","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2025\/04\/30\/data-analyst-or-data-engineer-or-analytics-engineer-or-bi-engineer\/","title":{"rendered":"Data Analyst or Data Engineer or Analytics Engineer or BI Engineer\u200a?"},"content":{"rendered":"<p>    Data Analyst or Data Engineer or Analytics Engineer or BI Engineer\u200a?<br \/>\n \t<BR><br \/>\n<BR><\/BR><br \/>\n    <!-- no image --><br \/>\n \t<BR><br \/>\n<BR><\/BR><\/p>\n<div>\n<p class=\"wp-block-paragraph\">If <mdspan datatext=\"el1745456944555\" class=\"mdspan-comment\">you\u2019ve followed<\/mdspan> me for a while, you probably know I started my career as a <strong>QA engineer<\/strong> before transitioning into the world of <strong>data analytics<\/strong>. I didn\u2019t go to school for it, didn\u2019t have a mentor, and didn\u2019t land in a formal training program. Everything I know today\u2014from SQL to modeling to storytelling with data\u2014is self-taught. And believe me, it\u2019s been a journey of trial, error, learning, and re-learning.<\/p>\n<h2 class=\"wp-block-heading\">The Dilemma That Changed My Career<\/h2>\n<p class=\"wp-block-paragraph\">A few years ago, I started thinking about switching organizations. Like many people in fast-evolving tech roles, I faced a surprisingly difficult question:<\/p>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">What role am I actually doing? Which roles should I apply for?<\/p>\n<\/blockquote>\n<p class=\"wp-block-paragraph\">On paper, I was a <strong>Data Analyst<\/strong>. But in reality, my role straddled several functions: writing SQL pipelines, building dashboards, defining KPIs, and digging into product analytics. I wasn\u2019t sure whether I should be applying for Analyst roles, BI roles, or something entirely different.<\/p>\n<p class=\"wp-block-paragraph\">To make things worse, back then, job titles were vague, and job descriptions were bloated with buzzwords. You\u2019d find a posting titled <em>\u201cData Analyst\u201d<\/em> that listed requirements like:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Build ML pipelines<\/li>\n<li class=\"wp-block-list-item\">Write complex ETL scripts<\/li>\n<li class=\"wp-block-list-item\">Maintain data lakes<\/li>\n<li class=\"wp-block-list-item\">Create dashboards<\/li>\n<li class=\"wp-block-list-item\">Present executive-level insights<\/li>\n<li class=\"wp-block-list-item\">And oh, by the way, be great at stakeholder management<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">It was overwhelming and confusing. And I know I\u2019m not alone in this.<\/p>\n<p class=\"wp-block-paragraph\">Fast forward to today: thankfully, things are evolving. There\u2019s still overlap between roles, but organizations have started to define them more clearly. In this article, I want to break down the <strong>real differences between data roles<\/strong>, through the lens of a real-world example.<\/p>\n<h3 class=\"wp-block-heading\">A Real-World Scenario: Meet <em>Quikee<\/em><br \/>\n<\/h3>\n<p class=\"wp-block-paragraph\">Let\u2019s imagine a fictional quick-commerce startup called <strong>Quikee<\/strong>, launching across multiple Indian cities. Their value proposition? Deliver groceries and essentials within <strong>10 minutes<\/strong>.<\/p>\n<p class=\"wp-block-paragraph\">Customers place orders through the app or website. Behind the scenes, there are micro-warehouses (also called \u201cdark stores\u201d) across cities, and a fleet of delivery partners who make those lightning-fast deliveries.<\/p>\n<p class=\"wp-block-paragraph\">Now, let\u2019s walk through the data needs of this company\u2014from the moment an order is placed, to the dashboards executives use in their Monday morning meetings.<\/p>\n<h3 class=\"wp-block-heading\">Step 1: Capturing and Storing Raw Data<\/h3>\n<p class=\"wp-block-paragraph\">The moment a customer places an order, <strong>transactional data<\/strong> is generated:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Timestamps<\/li>\n<li class=\"wp-block-list-item\">Order ID<\/li>\n<li class=\"wp-block-list-item\">Items ordered<\/li>\n<li class=\"wp-block-list-item\">Price<\/li>\n<li class=\"wp-block-list-item\">Discount codes<\/li>\n<li class=\"wp-block-list-item\">Customer location<\/li>\n<li class=\"wp-block-list-item\">Payment method<\/li>\n<li class=\"wp-block-list-item\">Assigned delivery partner<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Let\u2019s assume Quikee uses <strong>Amazon Kinesis<\/strong> to stream this data in real time to an <strong>S3 data lake<\/strong>. That stream is high-volume, time-sensitive, and crucial for business tracking.<\/p>\n<p class=\"wp-block-paragraph\">But here\u2019s the catch: raw data is messy. You can\u2019t use it directly for decision-making.<\/p>\n<p class=\"wp-block-paragraph\">So what happens next?<\/p>\n<h3 class=\"wp-block-heading\">Step 2: Building Data Pipelines<\/h3>\n<p class=\"wp-block-paragraph\">Enter the <strong>Data Engineers<\/strong>.<\/p>\n<p class=\"wp-block-paragraph\">They are responsible for:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Ingesting real-time data<\/li>\n<li class=\"wp-block-list-item\">Validating schema consistency<\/li>\n<li class=\"wp-block-list-item\">Handling failures and retries<\/li>\n<li class=\"wp-block-list-item\">Writing pipelines to move data from S3 into a data warehouse (say, Snowflake or Redshift)<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">This is where <strong>ETL<\/strong> (Extract, Transform, Load) or <strong>ELT<\/strong> pipelines come into play. Data engineers clean, format, and structure the data to make it queryable.<\/p>\n<p class=\"wp-block-paragraph\">For example, an order table might get split into:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">\n<strong>Orders<\/strong> \u2192 One row per order<\/li>\n<li class=\"wp-block-list-item\">\n<strong>Order_Items<\/strong> \u2192 One row per item in an order<\/li>\n<li class=\"wp-block-list-item\">\n<strong>Payments<\/strong> \u2192 One row per payment attempt<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">At this stage, raw logs are turned into structured tables that analysts can work with.<\/p>\n<h3 class=\"wp-block-heading\">Step 3: Dimensional Modeling &amp; OLAP<\/h3>\n<p class=\"wp-block-paragraph\">As leadership starts asking strategic questions like:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">\u201cWhich city brings in the most revenue?\u201d<\/li>\n<li class=\"wp-block-list-item\">\u201cWhich store is underperforming?\u201d<\/li>\n<li class=\"wp-block-list-item\">\u201cWhat\u2019s our average delivery time by zone?\u201d<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">\u2026it becomes clear that querying transactional data directly won\u2019t scale.<\/p>\n<p class=\"wp-block-paragraph\">That\u2019s where <strong>dimensional modeling<\/strong> comes in.<\/p>\n<p class=\"wp-block-paragraph\">Instead of flat, raw tables, data is structured into Fact and Dimension Tables.<\/p>\n<h3 class=\"wp-block-heading\">\n<img data-recalc-dims=\"1\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/1f538.png?ssl=1\" alt=\"\ud83d\udd38\" class=\"wp-smiley\" style=\"height: 1em; max-height: 1em;\"> <strong>Fact Tables<\/strong><br \/>\n<\/h3>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Large, quantitative data tables which contain foreign keys along with measures and metrics (<em>Well, most of the time. There are factless fact tables as well which do not have any measures<\/em>).<\/li>\n<li class=\"wp-block-list-item\">Examples: <code>fact_orders<\/code>, <code>fact_payments<\/code>, <code>fact_deliveries<\/code>\n<\/li>\n<li class=\"wp-block-list-item\">Contain metrics like revenue, order count, delivery time<\/li>\n<\/ul>\n<h3 class=\"wp-block-heading\">\n<img data-recalc-dims=\"1\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/1f539.png?ssl=1\" alt=\"\ud83d\udd39\" class=\"wp-smiley\" style=\"height: 1em; max-height: 1em;\"> <strong>Dimension Tables<\/strong><br \/>\n<\/h3>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Smaller, descriptive tables that help understand the data in a fact table<\/li>\n<li class=\"wp-block-list-item\">Examples: <code>dim_store<\/code>, <code>dim_product<\/code>, <code>dim_customer<\/code>, <code>dim_delivery_agent<\/code>\n<\/li>\n<li class=\"wp-block-list-item\">Help filter, group, and join facts for deeper insights<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">This structure enables <strong>OLAP<\/strong>\u2014fast, analytical querying across multiple dimensions. For example, you can now run queries like:<\/p>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">\u201cShow me average delivery time by store and hour of day, over the last 7 days.\u201d<\/p>\n<\/blockquote>\n<p class=\"wp-block-paragraph\">This step is done by Data Engineers at most of the organisations but I did build few Dim and Fact tables when I was working as a <a href=\"https:\/\/towardsdatascience.com\/tag\/business-intelligence\/\" title=\"Business Intelligence\">Business Intelligence<\/a> Engineer at Amazon.<\/p>\n<h3 class=\"wp-block-heading\">Step 4: Defining KPIs and Metrics<\/h3>\n<p class=\"wp-block-paragraph\">This is where <strong>Analytics Engineers (or BI Engineers)<\/strong> shine.<\/p>\n<p class=\"wp-block-paragraph\">They sit between the technical data layer and business users. Their responsibilities often include:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Defining KPIs (e.g., churn rate, repeat purchase %, time-to-fulfillment)<\/li>\n<li class=\"wp-block-list-item\">Writing logic for complex metrics (e.g., cohort retention, active users)<\/li>\n<li class=\"wp-block-list-item\">Creating <strong>semantic models<\/strong> or <strong>metrics layers<\/strong> in tools like dbt or Looker<\/li>\n<li class=\"wp-block-list-item\">Ensuring consistent definitions across the company<\/li>\n<\/ul>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">For example, at Amazon, our team didn\u2019t query raw data to calculate revenue every time. Instead, we created <strong>pre-aggregated fact tables<\/strong> at daily, weekly, and monthly grains. That way, dashboards loaded faster, and metrics stayed consistent across teams.<\/p>\n<\/blockquote>\n<p class=\"wp-block-paragraph\">Analytics Engineers act as translators between engineering and the business\u2014defining <strong>what<\/strong> we measure and <strong>how<\/strong> we measure it.<\/p>\n<h3 class=\"wp-block-heading\">Step 5: Analysis, Reporting &amp; Storytelling<\/h3>\n<p class=\"wp-block-paragraph\">Now comes the role of the <strong><a href=\"https:\/\/towardsdatascience.com\/tag\/data-analyst\/\" title=\"Data Analyst\">Data Analyst<\/a><\/strong>.<\/p>\n<p class=\"wp-block-paragraph\">Armed with clean, modeled data, they focus on answering real business questions like:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">\u201cWhy did retention drop in Bangalore last month?\u201d<\/li>\n<li class=\"wp-block-list-item\">\u201cWhich coupon codes drive the most new users?\u201d<\/li>\n<li class=\"wp-block-list-item\">\u201cWhat are the top products reordered in the first 30 days?\u201d<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">They build dashboards in tools like Tableau, Power BI, or Looker. They run ad-hoc SQL queries. They dive into A\/B test results, user behavior trends, and campaign effectiveness.<\/p>\n<p class=\"wp-block-paragraph\">But above all, they <strong>tell stories<\/strong> with data\u2014making complex numbers understandable and actionable for stakeholders.<\/p>\n<h3 class=\"wp-block-heading\">Who\u2019s\u00a0Who?<\/h3>\n<figure class=\"wp-block-image\"><img data-recalc-dims=\"1\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/cdn-images-1.medium.com\/max\/1600\/1%2AVa4pgbc8YPUc03l4-RAcVQ.png?ssl=1\" alt=\"\"><figcaption class=\"wp-element-caption\">Generated by\u00a0Author<\/figcaption><\/figure>\n<h3 class=\"wp-block-heading\">TL;DR: Where Do You Fit?<\/h3>\n<p class=\"wp-block-paragraph\">Here\u2019s how I think about it:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Love building robust pipelines and solving scalability problems? \u2192 You\u2019re a <strong>Data Engineer<\/strong>\n<\/li>\n<li class=\"wp-block-list-item\">Love defining business metrics and organizing complex datasets? \u2192 You\u2019re an <strong>Analytics Engineer<\/strong>\n<\/li>\n<li class=\"wp-block-list-item\">Love uncovering insights and storytelling with data? \u2192 You\u2019re a <strong>Data Analyst<\/strong>\n<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Of course, real-world roles often blend these. Especially at smaller companies, you may wear multiple hats. And that\u2019s okay.<\/p>\n<p class=\"wp-block-paragraph\">The key is not the title\u2014but <strong>where you add the most value<\/strong> and <strong>what energizes you<\/strong>.<\/p>\n<h2 class=\"wp-block-heading\">Final Thoughts<\/h2>\n<p class=\"wp-block-paragraph\">It took me a long time to understand what I actually do\u2014not just what my job title says. And if you\u2019ve ever felt that confusion, you\u2019re not alone.<\/p>\n<p class=\"wp-block-paragraph\">Today, I can clearly say I operate at the intersection of <strong>data modeling<\/strong>, <strong>business logic<\/strong>, and <strong>storytelling<\/strong>\u2014a sweet spot between analytics and engineering. And I\u2019ve learned that the ability to connect the dots is more important than fitting into a perfect box.<\/p>\n<p class=\"wp-block-paragraph\">If you\u2019ve walked a similar path\u2014or wear multiple hats in your role\u2014I\u2019d love to hear your story.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Drop a comment <img data-recalc-dims=\"1\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/1f447.png?ssl=1\" alt=\"\ud83d\udc47\" class=\"wp-smiley\" style=\"height: 1em; max-height: 1em;\"> or share this with someone figuring it out too.<\/strong><\/p>\n<p>The post <a href=\"https:\/\/towardsdatascience.com\/data-analyst-or-data-engineer-or-analytics-engineer-or-bi-engineer\/\">Data Analyst or Data Engineer or Analytics Engineer or BI Engineer\u200a?<\/a> appeared first on <a href=\"https:\/\/towardsdatascience.com\/\">Towards Data Science<\/a>.<\/p>\n<\/div>\n<p> \t<BR><br \/>\n <BR><\/BR><br \/>\n    Isha Garg<br \/>\n \t<BR><br \/>\n<BR><\/BR><br \/>\n<a href=\"https:\/\/towardsdatascience.com\/data-analyst-or-data-engineer-or-analytics-engineer-or-bi-engineer\/\">Go to original source<\/a><br \/>\n \t<BR><br \/>\n <BR><\/BR><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Data Analyst or Data Engineer or Analytics Engineer or BI Engineer\u200a? If you\u2019ve followed me for a while, you probably know I started my career as a QA engineer before transitioning into the world of data analytics. I didn\u2019t go to school for it, didn\u2019t have a mentor, and didn\u2019t land in a formal training [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[62,992,1668,2001,2512,401,83],"tags":[84,2128,2513],"class_list":["post-3452","post","type-post","status-publish","format-standard","hentry","category-aimldsaimlds","category-analytics-engineering","category-business-intelligence","category-data-analyst","category-data-roles","category-data-engineering","category-data-science","tag-data","tag-engineer","tag-roles"],"_links":{"self":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/3452"}],"collection":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/comments?post=3452"}],"version-history":[{"count":0,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/3452\/revisions"}],"wp:attachment":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/media?parent=3452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/categories?post=3452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/tags?post=3452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}