{"id":3767,"date":"2025-05-13T07:02:30","date_gmt":"2025-05-13T07:02:30","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2025\/05\/13\/pause-your-ml-pipelines-for-human-review-using-aws-step-functions-slack\/"},"modified":"2025-05-13T07:02:30","modified_gmt":"2025-05-13T07:02:30","slug":"pause-your-ml-pipelines-for-human-review-using-aws-step-functions-slack","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2025\/05\/13\/pause-your-ml-pipelines-for-human-review-using-aws-step-functions-slack\/","title":{"rendered":"Pause Your ML Pipelines for Human Review Using AWS Step Functions + Slack"},"content":{"rendered":"

Pause Your ML Pipelines for Human Review Using AWS Step Functions + Slack
\n \t

\n
<\/BR>
\n
\n \t

\n
<\/BR><\/p>\n

\n

Have you ever<\/mdspan> wanted to pause an automated workflow to wait for a human decision?<\/p>\n

Maybe you need approval before provisioning cloud resources, promoting a machine learning model to production, or charging a customer\u2019s credit card.<\/p>\n

In many\u00a0data science and machine learning workflows<\/strong>, automation gets you 90% of the way \u2014 but that critical last step often needs human judgment.<\/p>\n

Especially in production environments,\u00a0model retraining<\/strong>,\u00a0anomaly overrides<\/strong>, or\u00a0large data movements<\/strong>\u00a0require careful human review to avoid expensive mistakes.<\/p>\n

In my case, I needed to manually review situations where my system flagged\u00a0more than 6% of customer data for anomalies<\/strong>\u00a0\u2014 often due to accidental pushes by customers.<\/p>\n

Before I implemented a proper workflow, this was handled informally: developers would directly update production databases (!) \u2014 risky, error-prone, and unscalable.<\/p>\n

To solve this, I built a\u00a0scalable manual approval system<\/strong>\u00a0using\u00a0AWS<\/a> Step Functions<\/strong>,\u00a0Slack<\/strong>,\u00a0Lambda<\/strong>, and\u00a0SNS<\/strong>\u00a0\u2014 a cloud-native, low-cost architecture that cleanly paused workflows for human approvals without spinning up idle compute.<\/p>\n

In this post, I\u2019ll walk you through the full design, the AWS resources involved, and how you can apply it to your own critical workflows. <\/p>\n

Let\u2019s get into it \"\ud83d\udc47\"<\/p>\n

The Solution<\/h2>\n

My application is deployed in the AWS ecosystem, so we\u2019ll use Aws Step Functions<\/a> to build a state machine that:<\/p>\n

    \n
  1. Executes business logic<\/li>\n
  2. Lambda with\u00a0WaitForTaskToken<\/code>\u00a0to pause until approval<\/li>\n
  3. Sends a Slack message requesting approval (can be an email\/)<\/li>\n
  4. Waits for a human to click \u201cApprove\u201d or \u201cReject\u201d<\/li>\n
  5. Resumes automatically from the same point<\/li>\n<\/ol>\n
    \"\"
    The Step function flow<\/figcaption><\/figure>\n

    Here is a youtube video showing the demo and actual application in action:<\/p>\n

    \n
    \n