As data We can also handle words, json, xml feeds, and pictures of cats. But what about a cardboard box full of things like this? <\/p>\n The info on this receipt wants so badly to be in a tabular database somewhere. Wouldn\u2019t it be great if we could scan all these, run them through an LLM, and save the results in a table? <\/p>\n Lucky for us, we live in the era of Document Ai<\/a>. Document AI combines OCR with LLMs and allows us to build a bridge between the paper world and the digital database world.<\/p>\n All the major cloud vendors have some version of this\u2026 <\/p>\n Here I\u2019ll share my thoughts on Snowflake<\/a>\u2019s Document AI. Aside from using Snowflake at work, I have no affiliation with Snowflake. They didn\u2019t commission me to write this piece and I\u2019m not part of any ambassador program. All of that is to say I can write an unbiased<\/em> review of Snowflake\u2019s Document AI<\/a>.<\/p>\n Document AI allows users to quickly extract information from digital documents. When we say \u201cdocuments\u201d we mean pictures with words.\u00a0Don\u2019t confuse this with niche NoSQL things<\/a>. <\/p>\n The product combines OCR and LLM models so that a user can create a set of prompts and execute those prompts against a large collection of documents all at once. <\/p>\n LLMs and OCR both have room for error. Snowflake solved this by (1) banging their heads against OCR until it\u2019s sharp \u2014 I see you, Snowflake developer \u2014 and (2) letting me fine-tune my LLM.\u00a0<\/p>\n Fine-tuning the Snowflake LLM feels a lot more like glamping<\/a> than some rugged outdoor adventure. I review 20+ documents, hit the \u201ctrain model\u201d button, then rinse and repeat until performance is satisfactory. Am I even a data scientist anymore?<\/p>\n Once the model is trained, I can run my prompts on 1000 documents at a time. I like to save the results to a table but you could do whatever you want with the results real time.<\/p>\n This product is cool for several reasons.<\/p>\n Of course some elements are still built in house. Once I extract information from the document I have to figure out what to do with that information. That\u2019s relatively quick work, though.<\/p>\n I work at a company called IntelyCare<\/a>. We operate in the healthcare staffing space, which means we help hospitals, nursing homes, and rehab centers find quality clinicians for individual shifts, extended contracts, or full-time\/part-time engagements.\u00a0<\/p>\n Many of our facilities require clinicians to have an up-to-date flu shot. Last year, our clinicians submitted over 10,000 flu shots in addition to hundreds of thousands of other documents. We manually reviewed all of these manually to ensure validity. Part of the joy of working in the healthcare staffing world!<\/p>\n Spoiler Alert: Using Document AI, we were able to reduce the number of flu-shot documents needing manual review by ~50% and all in just a couple of weeks.<\/strong><\/p>\n To pull this off, we did the following:<\/p>\n For this project, false positives pose a business risk. We don\u2019t want to approve a document that\u2019s expired or missing key information. We kept iterating until the false-positive rate hit zero. We\u2019ll have some false positives eventually, but fewer than what we have now with a human review process. <\/p>\n False negatives, however, are harmless. If our pipeline doesn\u2019t like a flu shot, it simply routes the document to the human team for review.\u00a0If they go on to approve the document, it\u2019s business as usual. <\/p>\n The model does well with the clean\/easy documents, which account for ~50% of all flu shots. If it\u2019s messy or confusing, it goes back to the humans as before.\u00a0<\/p>\n Initially, our prompts attempted to determine validity of the document.<\/p>\n Bad: Is the document already expired?<\/em><\/p>\n We found it far<\/em> more effective to limit our prompts to questions that could be answered by looking at the document. The LLM doesn\u2019t determine<\/em> anything. It just grabs the relevant data points off the page.<\/p>\n Good: What is the expiration date?\u00a0<\/em><\/p>\n Save the results and do the math downstream. <\/p>\n We had a few duplicate flu shots from one clinician in our training data. Call this clinician Ben. One of our prompts was, \u201cwhat is the patient\u2019s name?\u201d Because \u201cBen\u201d was in the training data multiple times, any remotely unclear document would return with \u201cBen\u201d as the patient name. <\/p>\n So overfitting is still a thing. Over\/under sampling is still a thing. We tried again with a more thoughtful collection of training documents and things did much better.<\/p>\n Document AI is pretty magical, but not that<\/em> magical. Fundamentals still matter. <\/p>\n To my knowledge, Snowflake does not have a way to render the document image as an embedding<\/a>. You can create an embedding from the extracted text, but that won\u2019t tell you if the text was written by hand or not. As long as the text<\/em> is valid, the model and downstream logic will give it a green light.<\/p>\n You could fix this pretty easily by comparing image embeddings of submitted documents to the embeddings of accepted documents. Any document with an embedding way out in left field is sent back for human review. This is straightforward work, but you\u2019ll have to do it outside Snowflake for now.\u00a0<\/p>\n Snowflake has a reputation of being spendy. And for HIPAA compliance concerns we run a higher-tier Snowflake account for this project. I tend to worry about running up a Snowflake tab.<\/p>\n In the end we had to try extra hard to spend more than $100\/week while training the model. We ran thousands of documents through the model every few days to measure its accuracy while iterating on the model, but never managed to break the budget. <\/p>\n Better still, we\u2019re saving money on the manual review process. The costs for AI reviewing 1000 documents (approves ~500 documents) is ~20% of the cost we spend on humans reviewing the remaining 500. All in, a 40% reduction in costs for reviewing flu-shots.<\/p>\n I\u2019ve been impressed with how quickly we could complete a project of this scope using Document AI. We\u2019ve gone from months to days. I give it 4 stars out of 5, and am open to giving it a 5th star if Snowflake ever gives us access to image embeddings.\u00a0<\/p>\n Since flu shots, we\u2019ve deployed similar models for other documents with similar or better results. And with all this prep work, instead of dreading the upcoming flu season, we\u2019re ready to bring it on. <\/p>\n \n The post An Unbiased Review of Snowflake\u2019s Document AI<\/a> appeared first on Towards Data Science<\/a>.<\/p>\n<\/div>\n![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-46.png?ssl=1\")
![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/annie-spratt-recgFWxDO1Y-unsplash.jpg?ssl=1\")
\n
\nWhat is Document AI?\u00a0<\/h2>\n
![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-47-1024x430.png?resize=1024%2C430&ssl=1\")
\nWhy does it matter?\u00a0<\/h2>\n
\n
\n
\nOur Use Case \u2014 Bring on Flu Season:<\/h2>\n
\n
\n
\nThings we learned along the way<\/h2>\n
\n
\n
\n
\n
\nSumming up<\/h2>\n