Transforming John Holland’s Delivery Docket Processing with AI and AWS.
Overview
John Holland Group (JHG) is a leader in the construction and infrastructure industry, known for delivering large-scale, complex projects. While JHG’s docket management for large projects operates seamlessly through automated data exchange, smaller projects presented a unique challenge due to the reliance on paper and handwritten delivery dockets. The manual processes required for these dockets led to inefficiency, human error, and delays that impacted operational and financial reporting.
To overcome these hurdles, Mantalus partnered with JHG to develop a Proof of Concept (PoC) using AI and AWS services. The PoC was instrumental in proving that an AI-driven approach was both capable and suitable for a full-scale solution, and ultimately demonstrated how automation of data ingestion from paper and handwritten dockets could improve accuracy and efficiency in docket processing.
Key Solution Elements
Mantalus’ custom AWS infrastructure solution
Enhanced Data Ingestion
Support for API Gateway and email uploads
User Friendly UI
A Forms Manager interface for configuration, manual reviews, and reporting.
End-to-End Automation
Event-driven pipelines initiated by S3 uploads. Each major step in the process is logged to a ‘Ledger’ database, enabling detailed reporting, monitoring, and enabling restarts where required.
Infrastructure as Code
Deployment using AWS Cloud Development Kit (CDK) for consistency and scalability.
Challenge
Addressing Inefficient Manual Workflows
Delivery dockets, key documents used for tracking materials and deliveries, were manually transcribed into spreadsheets. These dockets are critical for reconciling purchase orders and invoices and accurately assigning costs to specific cost centers. However, the manual transcription process was both inefficient and prone to errors, which frequently disrupted these downstream processes and impacted financial reporting and resource management.
JHG sought a scalable Intelligent Document Processing (IDP) solution that could:
- Automate data extraction from delivery dockets.
- Categorize and map data fields with high accuracy.
- Integrate seamlessly with downstream systems to support reporting and decision-making.
Solution
Leveraging AI on AWS
Mantalus designed a POC solution leveraging AWS’s cloud services to address JHG’s challenges. The goal was to automate the ingestion, classification, and mapping of delivery docket data while ensuring scalability, cost-effectiveness, and ease of integration
Key Components of the POC Workflow
- Image Ingestion and Preprocessing – Delivery dockets were uploaded to an Amazon S3 bucket, triggering a Lambda function to preprocess images. Enhancements included de-skewing and super-resolution to optimize image quality for text extraction.
- Data Extraction with AWS Textract – AWS Textract was used to extract text and structured data such as tables and forms, generating a JSON output for further processing.
- Identifying Known Document Types – Known docket types were matched using document feature embedding vectors, enabling the system to identify patterns and similarities across dockets for efficient processing.
- AI-Driven Document Classification – A multimodal call to AWS Bedrock was used to classify dockets based on layout and content, enabling accurate categorization for downstream processing.
- Field Mapping and Validation – Extracted data fields were mapped according to predefined categories using a multimodal call to AWS Bedrock. Validation logic ensured high accuracy before dispatching the data.
- Data Dispatch and Integration – The mapped data was stored in a structured format for integration with JHG’s existing downstream systems.
Designing the Full IDP Solution: Building on POC Learnings
The Full IDP Solution phase, guided by the learnings from the POC, aims to deliver a full-scale production-ready IDP solution with enhanced capabilities:
Key Features of the MVP
- Enhanced Data Ingestion: Support for API Gateway and email uploads.
- User-Friendly UI: A Forms Manager interface for configuration, manual reviews, and reporting.
- End-to-End Automation: Event-driven pipelines initiated by S3 uploads. Each major step in the process is logged to a ‘Ledger’ database, enabling detailed reporting, monitoring, and enabling restarts where required.
- Infrastructure as Code: Deployment using AWS Cloud Development Kit (CDK) for consistency and scalability.
Challenges
- Image Quality Variability: Advanced preprocessing techniques addressed issues like low resolution and poor lighting, as well as challenges such as low-contrast handwriting on carbon copies, skewed camera angles, and rotated images. These improvements ensured higher accuracy during text extraction and classification.
- Handling Unknown Docket Types: New or unknown docket types were identified using a pattern-matching confidence score, ensuring they were flagged for manual review. This approach prevented misclassification and allowed the system to adapt dynamically to previously unseen formats.
- Form Field Ambiguity: When similar fields appeared across different docket types, generative AI models were leveraged to infer the most likely field mapping. This reduced ambiguity, ensured accurate categorization, and supported efficient downstream processing.
- Data Content Variability: Textract’s structured representation, combined with confidence thresholds and feature metadata, enabled reliable identification and handling of variable repeating content such as line items. For new or unknown docket types, Bedrock’s flexibility supported initial categorization and column mapping through generative AI, simplifying the manual review process.
- Scalability Requirements: Containerized Lambda functions ensured the solution could handle high workloads efficiently while also solving packaging challenges for the complex and large libraries required.
Outcomes
Key outcomes included:
Faster Processing: Significant reductions in manual data entry time by automating extraction and classification, streamlining workflows, and enabling faster turnaround for operational tasks.
Improved Accuracy: Minimized human error through automated data extraction, classification, and AI-driven validation processes, improving confidence in reconciliations.
Increased Coverage for Variability: Successfully handled a wide range of docket formats, including clear handwritten dockets, printed dockets, and structured forms, demonstrating adaptability to real-world variability.
Cost Optimization: Leveraged AWS’s pay-as-you-go model for efficient resource usage and reduced the manual effort required for repetitive and time-consuming tasks.
Scalable Architecture: Validated the capability of the AWS-powered solution to process high volumes of dockets while maintaining performance, reliability, and cost efficiency.
Actionable Insights: Provided structured outputs ready for downstream integration, enabling improved reporting and decision-making capabilities.
The POC successfully automated the docket processing workflow, proving the feasibility and suitability of a data-centric AI-driven automation approach.
Why Mantalus?
Mantalus’ capability is built on being the tip of the spear to solve difficult and unique problems with AWS technology. Our strength is based on having consultants with an array of industry experience, who have themselves faced a litany of complex, business critical, technology roadblocks; and found creative and class-leading ways to solve them.
We’re great at developing AWS centric architectures, where none have existed before. Think platforms or middleware that have never been attempted on AWS by anyone; or even completely AWS native solutions, removing the need for expensive 3rd party solutions.
So, where can we be useful and important to you? Anywhere there’s pain.
When you’re scratching your heads with a tricky use case that doesn’t fit neatly into a known solution or reference architecture…. think Mantalus. We’re the cure!
AWS has a fantastic array of services – and if you partner with Mantalus we can use them to help you solve just about anything.