# Avoid Docker Hub Rate Limit with Secure Credential Management: Meet _sm-docker-credential-helper_

Docker Hub’s recent enforcement of pull rate limits—100 pulls per 6 hours for anonymous users and 200 for authenticated free-tier users—has created friction for developers and DevOps teams. This is especially problematic for CI/CD pipelines, where multiple jobs may share the same outbound IP address.

For organisations using self-hosted runners (e.g., GitHub Actions, GitLab CI, Jenkins, etc.) behind a shared NAT gateway, these limits can be hit quickly, even if individual jobs aren’t pulling many images. To avoid hitting those limits and securely manage credentials, the **_sm-docker-credential-helper_** offers a secure, centralised solution using AWS Secrets Manager.

## What is sm-docker-credential-helper ?

**_sm-docker-credential-helper_** is a Docker credential helper that retrieves Docker Hub credentials directly from AWS Secrets Manager, allowing you to:

- Authenticate Docker pulls
- Avoid rate limits
- Store secrets securely
- Share a single set of credentials across environments

## Why Is This Critical Now ?

**Docker Hub Pull Rate Limits**

- _**100 pulls/6hr**_ for anonymous users
- **_200 pulls/6hr_** for free-tier authenticated users (per IP address)

If you’re using **_self-hosted CI runners sharing a NAT IP_**, Docker sees all requests as coming from one user, regardless of how many machines or containers are pulling. This means:

Even a modest pipeline with multiple stages can hit Docker Hub’s rate limits quickly—causing builds to fail unexpectedly.

With _sm-docker-credential-helper_, you can authenticate each request using **centralised**, **securely stored credentials**, ensuring rate limits are applied per **authenticated account** rather than shared IP.

## How **_sm-docker-credential-helper_** Works

### Prerequisites

- AWS credentials configured locally or on the runner
- Docker credentials stored in AWS Secrets Manager:

```json

```

### Setup Steps

1. Build the Helper (requires Golang installed locally)

```bash
   git clone https://github.com/vhoanguyen/sm-docker-credentials-helper
   cd sm-docker-credentials-helper
   VERSION=1.0.0 make build
   ```

2. Install the Binary to your $PATH

```bash
   cp ./bin/sm-login-linux-amd64 /usr/local/bin/docker-credential-sm-login
   ```

3. Configure Docker Update _**~/.docker/config.json**_:

```json

```

4. Create CFN Stack Resource

```yaml
   AWSTemplateFormatVersion: '2010-09-09'
   Resources:
     MySecret:
       Type: AWS::SecretsManager::Secret
       Properties:
         Name: MySecret
         Description: "A sample secret for demonstration purposes"
         SecretString: |
   
         Tags:
           - Key: Environment
             Value: Production
         ResourcePolicy:
           Version: "2012-10-17"
   ```

5. Set Required Environment Variables

```bash
   export DOCKER_SECRET_NAME=MySecret
   export AWS_PROFILE=your-aws-profile
   ```

6. Create Docker Organisation or Personal Token and Update Secret Manager

- [https://docs.docker.com/security/for-admins/access-tokens/](https://docs.docker.com/security/for-admins/access-tokens/)
   - [https://docs.docker.com/security/for-developers/access-tokens/](https://docs.docker.com/security/for-developers/access-tokens/)

## Perform simple tests

Test 1: Access to Secret Manager Resource

```bash
AWS_PROFILE=YOUR_AWS_PROFILE \ 
DOCKER_SECRET_NAME=MySecret \ 
/usr/local/bin/docker-credential-sm-login list
```

Test 2: Docker Login

```bash
echo https://index.docker.io/v1/ | AWS_PROFILE=YOUR_AWS_PROFILE DOCKER_SECRET_NAME=MySecret \
/usr/local/bin/docker-credential-sm-login get
```

Test 2: Run Hello World Image (check **_~/.sm/_** for debug logs )

```bash
AWS_PROFILE=YOUR_AWS_PROFILE \
DOCKER_SECRET_NAME=MySecret \
docker pull hello-world
```

## Why Use This in self-hosted CI/CD

If you’re running pipelines with tools like:

- GitHub Actions (self-hosted)
- GitLab Runners
- Jenkins
- ArgoCD or custom Kubernetes jobs

…and all your runners exit through _**one shared IP**_, you’re highly vulnerable to rate-limiting—even with authenticated users.

By using **_sm-docker-credential-helper_**:

- All runners authenticate with valid Docker credentials
- You avoid being rate-limited due to shared IP
- Secrets are centrally stored, easily rotated, and not exposed in plain text

## Benefits Recap

✅ Bypass Docker rate limits by authenticating every pull

✅ Avoid pipeline failures caused by NAT-based rate limits

✅ Secure Docker credentials using AWS Secrets Manager

✅ Easily share credentials across multiple machines/runners

✅ No local plaintext config—credentials are pulled on demand

[Hoa Nguyen](/content/author/vanhoa-nguyenmantalus-com/index.html) 2025-05-21T15:14:37+10:00