{"id":3248,"date":"2025-04-22T07:02:44","date_gmt":"2025-04-22T07:02:44","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2025\/04\/22\/building-a-personal-api-for-your-data-projects-with-fastapi\/"},"modified":"2025-04-22T07:02:44","modified_gmt":"2025-04-22T07:02:44","slug":"building-a-personal-api-for-your-data-projects-with-fastapi","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2025\/04\/22\/building-a-personal-api-for-your-data-projects-with-fastapi\/","title":{"rendered":"Building a Personal API for Your Data Projects with FastAPI"},"content":{"rendered":"<p>    Building a Personal API for Your Data Projects with FastAPI<br \/>\n \t<BR><br \/>\n<BR><\/BR><br \/>\n    <!-- no image --><br \/>\n \t<BR><br \/>\n<BR><\/BR><\/p>\n<div>\n<p class=\"wp-block-paragraph\"><mdspan datatext=\"el1745298385345\" class=\"mdspan-comment\">How many times<\/mdspan> have you had a messy Jupyter Notebook filled with copy-pasted code just to re-use some data wrangling logic? Whether you do it for passion or for work, if you code a lot, then you\u2019ve probably answered something like \u201cway too many\u201d.<\/p>\n<p class=\"wp-block-paragraph\">You\u2019re not alone.<\/p>\n<p class=\"wp-block-paragraph\">Maybe you tried to share data with colleagues or plugging your latest ML model into a slick dashboard, but sending CSVs or rebuilding the dashboard from scratch doesn\u2019t feel correct.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Here\u2019s today\u2019s fix (and topic): build yourself a personal API.<\/strong><br \/>In this post, I\u2019ll show you how to set up a lightweight, powerful FastAPI service to expose your datasets or models and\u00a0<em>finally<\/em>\u00a0give your data projects the modularity they deserve.<\/p>\n<p class=\"wp-block-paragraph\">Whether you\u2019re a solo <a href=\"https:\/\/towardsdatascience.com\/tag\/data-science\/\" title=\"Data Science\">Data Science<\/a> enthusiast, a student with side projects, or a seasoned ML engineer, this is for you.<\/p>\n<p class=\"wp-block-paragraph\">And no, I\u2019m not being paid to promote this service. It\u2019d be good, but the reality is far from that. I just happen to enjoy using it and I thought it was worth being shared.<\/p>\n<p class=\"wp-block-paragraph\">Let\u2019s review today\u2019s table of contents:<\/p>\n<ol class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">What is a personal API? (And why should you care?)<\/li>\n<li class=\"wp-block-list-item\">Some use cases<\/li>\n<li class=\"wp-block-list-item\">Setting it up with <a href=\"https:\/\/towardsdatascience.com\/tag\/fastapi\/\" title=\"Fastapi\">Fastapi<\/a>\n<\/li>\n<li class=\"wp-block-list-item\">Conclusion<\/li>\n<\/ol>\n<h2 class=\"wp-block-heading\">What Is a Personal API? (And Why Should You Care?)<\/h2>\n<p class=\"wp-block-paragraph\">99% of people reading this will already be familiar with the API concept. But for that 1%, here\u2019s a brief intro that will be complemented with code in the next sections:<\/p>\n<p class=\"wp-block-paragraph\">An\u00a0<strong>API<\/strong>\u00a0(Application Programming Interface) is a set of rules and tools that allows different software applications to communicate with each other. It defines\u00a0<strong>what you can ask a program to do<\/strong>, such as \u201cgive me the weather forecast\u201d or \u201csend a message.\u201d And that program handles the request behind the scenes and returns the result.<\/p>\n<p class=\"wp-block-paragraph\">So, what is a <strong>personal API<\/strong>? It\u2019s essentially a small web service that exposes your data or logic in a structured, reusable way. Think of it like a mini app that responds to HTTP requests with JSON versions of your data.<\/p>\n<p class=\"wp-block-paragraph\">Why would that be a good idea? In my opinion, it has different advantages:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">As already mentioned, <strong>reusability<\/strong>. We can use it from our Notebooks, dashboards or scripts without having to rewrite the same code several times.<\/li>\n<li class=\"wp-block-list-item\">\n<strong>Collaboration<\/strong>: your teammates can easily access your data through the API endpoints without needing to duplicate your code or download the same datasets in their machines.<\/li>\n<li class=\"wp-block-list-item\">\n<strong>Portability<\/strong>: You can deploy it anywhere\u2014locally, on the cloud, in a container, or even on a Raspberry Pi.<\/li>\n<li class=\"wp-block-list-item\">\n<strong>Testing<\/strong>: Need to test a new feature or model update? Push it to your API and instantly test across all clients (notebooks, apps, dashboards).<\/li>\n<li class=\"wp-block-list-item\">\n<strong>Encapsulation and Versioning<\/strong>: You can version your logic (v1, v2, etc.) and separate raw data from processed logic cleanly. That\u2019s a huge plus for maintainability.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">And FastAPI is perfect for this. But let\u2019s see some real use cases where anyone like you and me would benefit from a personal API.<\/p>\n<h2 class=\"wp-block-heading\">Some Use Cases<\/h2>\n<p class=\"wp-block-paragraph\">Whether you\u2019re a data scientist, analyst, ML engineer, or just building cool stuff on weekends, a personal API can become your secret productivity weapon. Here are three examples:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">\n<strong>Model-as-a-service<\/strong> (MASS): train an ML model locally and expose it to your public through an endpoint like <code>\/predict<\/code>. And options from here are endless: rapid prototyping, integrating it on a frontend\u2026<\/li>\n<li class=\"wp-block-list-item\">\n<strong>Dashboard-ready data<\/strong>: Serve preprocessed, clean, and filtered datasets to BI tools or custom dashboards. You can centralize logic in your API, so the dashboard stays lightweight and doesn\u2019t re-implement filtering or aggregation.<\/li>\n<li class=\"wp-block-list-item\">\n<strong>Reusable data access layer<\/strong>: When working on a project that contains multiple Notebooks, has it ever happened to you that the first cells on all of them contain always the same code? Well, what if you centralized all that code into your API and got it done from a single request? Yes, you could modularize it as well and call a function to do the same, but creating the API allows you to go one step further, being able to use it easily from anywhere (not just locally).<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">I hope you get the point. Options are endless, just like its usefulness.<\/p>\n<p class=\"wp-block-paragraph\">But let\u2019s get to the interesting part: building the API.<\/p>\n<h2 class=\"wp-block-heading\">Setting it up with FastAPI<\/h2>\n<p class=\"wp-block-paragraph\">As always, start by setting up the environment with your favorite env tool (venv, pipenv\u2026). Then, install fastapi and uvicorn with <code>pip install fastapi uvicorn<\/code>. Let\u2019s understand what they do:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">\n<strong>FastAPI<\/strong>[1]: it\u2019s the library that will allow us to develop the API, essentially.<\/li>\n<li class=\"wp-block-list-item\">\n<strong>Uvicorn<\/strong>[2]: it\u2019s what will allow us to run the web server.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Once installed, we only need one file. For simplicity, we\u2019ll call it <em>app.py<\/em>.<\/p>\n<p class=\"wp-block-paragraph\">Let\u2019s now put some context into what we\u2019ll do: Imagine we\u2019re building a smart irrigation system for our vegetable garden at home. The irrigation system is quite simple: we have a moisture sensor that reads the soil moisture with certain frequency, and we want to activate the system when it\u2019s below 30%. <\/p>\n<p class=\"wp-block-paragraph\">Of course we want to automate it locally, so when it hits the threshold it starts dropping water. But we\u2019re also interested in being able to access the system remotely, maybe reading the current value or even triggering the water pump if we want to. That\u2019s when the personal API can come in handy.<\/p>\n<p class=\"wp-block-paragraph\">Here\u2019s the basic code that will allow us to do just that (note that I\u2019m using another library, <strong>duckdb<\/strong>[3], because that\u2019s where I would store the data \u2014 but you could just use sqlite3, pandas, or whatever you like):<\/p>\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-julia\">\n\nimport datetime\n\nfrom fastapi import FastAPI, Query\nimport duckdb\n\napp = FastAPI()\nconn = duckdb.connect(\"moisture_data.db\")\n\n@app.get(\"\/last_moisture\")\ndef get_last_moisture():\n    query = \"SELECT * FROM moisture_reads ORDER BY day DESC, time DESC LIMIT 1\"\n    return conn.execute(query).df().to_dict(orient=\"records\")\n\n@app.get(\"\/moisture_reads\/{day}\")\ndef get_moisture_reads(day: datetime.date, time: datetime.time = Query(None)):\n    query = \"SELECT * FROM moisture_reads WHERE day = ?\"\n    args = [day]\n    if time:\n        query += \" AND time = ?\"\n        args.append(time)\n    \n    return conn.execute(query, args).df().to_dict(orient=\"records\")\n\n@app.get(\"\/trigger_irrigation\")\ndef trigger_irrigation():\n    # This is a placeholder for the actual irrigation trigger logic\n    # In a real-world scenario, you would integrate with your irrigation system here\n    return {\"message\": \"Irrigation triggered\"}<\/code><\/pre>\n<p class=\"wp-block-paragraph\">Reading vertically, this code separates three main blocks:<\/p>\n<ol class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Imports<\/li>\n<li class=\"wp-block-list-item\">Setting up the app object and the DB connection<\/li>\n<li class=\"wp-block-list-item\">Creating the API endpoints<\/li>\n<\/ol>\n<p class=\"wp-block-paragraph\">1 and 2 are pretty straightforward, so we\u2019ll focus on the third one. What I did here was create 3 endpoints with their own functions:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">\n<code>\/last_moisture<\/code> shows the last sensor value (the most recent one).<\/li>\n<li class=\"wp-block-list-item\">\n<code>\/moisture_reads\/{day}<\/code> is useful to see the sensor reads from a single day. For example, if I wanted to compare moisture levels in winter with the ones in summer, I would check what\u2019s in <code>\/moisture_reads\/2024-01-01<\/code> and observe the differences with <code>\/moisture_reads\/2024-08-01<\/code>.<br \/>But I\u2019ve also made it able to read GET parameters if I\u2019m interested in checking a specific time. For example: <code>\/moisture_reads\/2024-01-01?time=10:00<\/code>\n<\/li>\n<li class=\"wp-block-list-item\">\n<code>\/trigger_irrigation<\/code> would do what the name suggests.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">So we\u2019re only missing one part, starting the server. See how simple it is to run it locally:<\/p>\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-bash\">uvicorn app:app --reload<\/code><\/pre>\n<p class=\"wp-block-paragraph\">Now I could visit:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">\n<a href=\"http:\/\/localhost:8000\/last_moisture\">http:\/\/localhost:8000\/last_moisture<\/a>\u00a0to see my last moisture<\/li>\n<li class=\"wp-block-list-item\">\n<a href=\"http:\/\/localhost:8000\/moisture_reads\/2024-01-01\">http:\/\/localhost:8000\/moisture_reads\/2024-01-01<\/a> to see the moisture levels of January 1st, 2024.<\/li>\n<li class=\"wp-block-list-item\">\n<a href=\"http:\/\/localhost:8000\/trigger_irrigation\">http:\/\/localhost:8000\/trigger_irrigation<\/a>\u00a0to start pumping water.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">But it doesn\u2019t end here. FastAPI provides another endpoint which is found in <a href=\"http:\/\/localhost:8000\/docs\">http:\/\/localhost:8000\/docs<\/a> that shows autogenerated interactive documentation for our API. In our case:<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"788\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-88-1024x788.png?resize=1024%2C788&#038;ssl=1\" alt=\"\" class=\"wp-image-601936\"><\/figure>\n<p class=\"wp-block-paragraph\">It\u2019s extremely useful when the API is collaborative, because we don\u2019t need to check the code to be able to see all the endpoints we have access to!<\/p>\n<p class=\"wp-block-paragraph\">And with just a few lines of code, very few in fact, we\u2019ve been able to build our personal API. It can obviously get a lot more complicated (and probably should) but that wasn\u2019t today\u2019s purpose.<\/p>\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n<p class=\"wp-block-paragraph\">With just a few lines of <a href=\"https:\/\/towardsdatascience.com\/tag\/python\/\" title=\"Python\">Python<\/a> and the power of FastAPI, you\u2019ve now seen how easy it is to expose your data or logic through a personal API. Whether you\u2019re building a smart irrigation system, exposing a machine learning model, or just tired of rewriting the same wrangling logic across notebooks\u2014this approach brings modularity, collaboration, and scalability to your projects.<\/p>\n<p class=\"wp-block-paragraph\">And this is just the beginning. You could:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Add authentication and versioning<\/li>\n<li class=\"wp-block-list-item\">Deploy to the cloud or a Raspberry Pi<\/li>\n<li class=\"wp-block-list-item\">Chain it to a frontend or a Telegram bot<\/li>\n<li class=\"wp-block-list-item\">Turn your portfolio into a living, breathing project hub<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">If you\u2019ve ever wanted your data work to\u00a0<em>feel<\/em>\u00a0like a real product\u2014this is your gateway.<\/p>\n<p class=\"wp-block-paragraph\">Let me know if you build something cool with it. Or even better, send me the URL to your\u00a0<code>\/predict<\/code>,\u00a0<code>\/last_moisture<\/code>, or whatever API you\u2019ve made. I\u2019d love to see what you come up with.<\/p>\n<h2 class=\"wp-block-heading\">Resources<\/h2>\n<p class=\"wp-block-paragraph\">[1] Ram\u00edrez, S. (2018).\u00a0<em>FastAPI<\/em>\u00a0(Version 0.109.2) [Computer software].\u00a0<a class=\"\">https:\/\/fastapi.tiangolo.com<\/a><\/p>\n<p class=\"wp-block-paragraph\">[2] Encode. (2018).\u00a0<em>Uvicorn<\/em>\u00a0(Version 0.27.0) [Computer software].\u00a0<a class=\"\" href=\"https:\/\/www.uvicorn.org\/\">https:\/\/www.uvicorn.org<\/a><\/p>\n<p class=\"wp-block-paragraph\">[3] M\u00fchleisen, H., Raasveldt, M., &amp; DuckDB Contributors. (2019).\u00a0<em>DuckDB<\/em>\u00a0(Version 0.10.2) [Computer software].\u00a0<a class=\"\" href=\"https:\/\/duckdb.org\/\">https:\/\/duckdb.org<\/a><\/p>\n<p>The post <a href=\"https:\/\/towardsdatascience.com\/building-a-personal-api-for-your-data-projects-with-fastapi\/\">Building a Personal API for Your Data Projects with FastAPI<\/a> appeared first on <a href=\"https:\/\/towardsdatascience.com\/\">Towards Data Science<\/a>.<\/p>\n<\/div>\n<p> \t<BR><br \/>\n <BR><\/BR><br \/>\n    Pol Marin<br \/>\n \t<BR><br \/>\n<BR><\/BR><br \/>\n<a href=\"https:\/\/towardsdatascience.com\/building-a-personal-api-for-your-data-projects-with-fastapi\/\">Go to original source<\/a><br \/>\n \t<BR><br \/>\n <BR><\/BR><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Building a Personal API for Your Data Projects with FastAPI How many times have you had a messy Jupyter Notebook filled with copy-pasted code just to re-use some data wrangling logic? Whether you do it for passion or for work, if you code a lot, then you\u2019ve probably answered something like \u201cway too many\u201d. You\u2019re [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[62,83,942,1727,157],"tags":[702,84,2423],"class_list":["post-3248","post","type-post","status-publish","format-standard","hentry","category-aimldsaimlds","category-data-science","category-fastapi","category-project-management","category-python","tag-api","tag-data","tag-personal"],"_links":{"self":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/3248"}],"collection":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/comments?post=3248"}],"version-history":[{"count":0,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/3248\/revisions"}],"wp:attachment":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/media?parent=3248"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/categories?post=3248"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/tags?post=3248"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}