> ## Documentation Index
> Fetch the complete documentation index at: https://private-7c7dfe99-vortex-format.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> Step-by-step guide on how to set up Amazon RDS MariaDB as a source for ClickPipes

# RDS MariaDB source setup guide

export const IAMAuthentication = ({engine, service, children}) => {
  const services = {
    aurora: {
      name: 'Aurora',
      resource: 'cluster',
      id: 'cluster-xxxxxxxxxxxxxx'
    },
    rds: {
      name: 'RDS',
      resource: 'instance',
      id: 'db-xxxxxxxxxxxxxx'
    }
  };
  const createUserStatements = {
    postgres: `CREATE USER clickpipes_iam_user;
GRANT rds_iam TO clickpipes_iam_user;`,
    mysql: `CREATE USER 'clickpipes_iam_user' IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS';`
  };
  const svc = services[String(service).toLowerCase()];
  const createUserSql = createUserStatements[String(engine).toLowerCase()];
  if (!svc) throw new Error(`Unsupported IAM authentication service: ${service}`);
  if (!createUserSql) throw new Error(`Unsupported IAM authentication engine: ${engine}`);
  return <>
      <p>
        Instead of a password, you can authenticate the ClickPipes user with an AWS IAM role. This lets ClickPipes connect to your Amazon {svc.name} {svc.resource} without storing database credentials.
      </p>

      <h4 id="enable-iam-authentication">Enable IAM authentication</h4>

      <ol>
        <li>Log in to your AWS account and go to the {svc.name} {svc.resource} you want to configure.</li>
        <li>Click <strong>Modify</strong>.</li>
        <li>Scroll to the <strong>Database authentication</strong> section.</li>
        <li>Select <strong>Password and IAM database authentication</strong>.</li>
        <li>Click <strong>Continue</strong>.</li>
        <li>Review the changes and select <strong>Apply immediately</strong>.</li>
      </ol>

      <h4 id="create-database-user">Create the ClickPipes user</h4>

      <p>Create the ClickPipes user with IAM authentication enabled, then grant it the same schema and replication privileges shown above:</p>

      <CodeBlock language="sql">{createUserSql}</CodeBlock>

      {children}

      <h4 id="obtaining-the-clickhouse-service-iam-role-arn">Obtain the ClickHouse service IAM role ARN</h4>

      <ol>
        <li>Log in to your ClickHouse Cloud account.</li>
        <li>Select the ClickHouse service you want to connect.</li>
        <li>Select the <strong>Settings</strong> tab.</li>
        <li>Scroll to the <strong>Network security information</strong> section at the bottom of the page.</li>
        <li>Copy the service's <strong>Service role ID (IAM)</strong> value, shown below.</li>
      </ol>

      <Frame>
        <img src="/images/cloud/security/secures3_arn.webp" alt="Service role ID (IAM) value in the Network security information section" />
      </Frame>

      <p>This value is your <code>{'{ClickHouse_IAM_ARN}'}</code> — the role ClickPipes uses to access your {svc.name} {svc.resource}.</p>

      <h4 id="obtaining-the-rds-resource-id">Obtain the resource ID</h4>

      <ol>
        <li>Log in to your AWS account and go to the {svc.name} {svc.resource} you want to configure.</li>
        <li>Select the <strong>Configuration</strong> tab.</li>
        <li>Note the <strong>Resource ID</strong> value — it looks like <code>{svc.id}</code>. This is your <code>{'{RDS_RESOURCE_ID}'}</code>, which you reference in the permissions policy.</li>
      </ol>

      <h4 id="manually-create-iam-role">Create the IAM role</h4>

      <ol>
        <li>Log in to your AWS account with an IAM user that has permission to create and manage IAM roles.</li>
        <li>Open the IAM console.</li>
        <li>
          Create a new IAM role with the following trust and permissions policies.

          <p>Trust policy (replace <code>{'{ClickHouse_IAM_ARN}'}</code> with the IAM role ARN of your ClickHouse instance):</p>

          <CodeBlock language="json">{`{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "{ClickHouse_IAM_ARN}"
      },
      "Action": [
        "sts:AssumeRole",
        "sts:TagSession"
      ]
    }
  ]
}`}</CodeBlock>

          <p>Permissions policy (replace <code>{'{RDS_RESOURCE_ID}'}</code> with the resource ID of your {svc.name} {svc.resource}, <code>{'{RDS_REGION}'}</code> with its region, and <code>{'{AWS_ACCOUNT}'}</code> with your AWS account ID):</p>

          <CodeBlock language="json">{`{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "rds-db:connect"
      ],
      "Resource": [
        "arn:aws:rds-db:{RDS_REGION}:{AWS_ACCOUNT}:dbuser:{RDS_RESOURCE_ID}/clickpipes_iam_user"
      ]
    }
  ]
}`}</CodeBlock>
        </li>
        <li>Once the role is created, copy its ARN. This is your <code>{'{RDS_ACCESS_IAM_ROLE_ARN}'}</code>.</li>
      </ol>

      <p>You can now use this IAM role to authenticate with your {svc.name} {svc.resource} from ClickPipes.</p>
    </>;
};

export const Image = ({img, alt, size = "lg"}) => {
  const normalizedSize = ["sm", "md", "lg"].includes(size) ? size : "lg";
  return <div className={`ch-image-${normalizedSize}`}>
      <Frame>
        <img src={img} alt={alt} />
      </Frame>
    </div>;
};

This is a step-by-step guide on how to configure your RDS MariaDB instance for replicating its data via the MySQL ClickPipe.

<br />

<Info>
  We also recommend going through the MySQL FAQs [here](/integrations/clickpipes/mysql/faq). The FAQs page is being actively updated.
</Info>

<h2 id="enable-binlog-retention-rds">
  Enable binary log retention
</h2>

The binary log is a set of log files that contain information about data modifications made to a MySQL server instance. Binary log files are required for replication. Both of the steps below must be followed:

<Steps>
  <Step title="Enable binary logging via automated backup" id="enable-binlog-logging-rds">
    The automated backups feature determines whether binary logging is turned on or off for MySQL. It can be set in the AWS console:

    <Image img="https://mintcdn.com/private-7c7dfe99-vortex-format/Kj4wT80mHNH7Aczn/images/integrations/data-ingestion/clickpipes/mysql/source/rds/rds-backups.webp?fit=max&auto=format&n=Kj4wT80mHNH7Aczn&q=85&s=5755c5b765df5950039f011a8dedc0b3" alt="Enabling automated backups in RDS" size="lg" border width="3230" height="530" data-path="images/integrations/data-ingestion/clickpipes/mysql/source/rds/rds-backups.webp" />

    Setting backup retention to a reasonably long value depending on the replication use-case is advisable.
  </Step>

  <Step title="Binlog retention hours" id="binlog-retention-hours-rds">
    Amazon RDS for MariaDB has a different method of setting binlog retention duration, which is the amount of time a binlog file containing changes is kept. If some changes aren't read before the binlog file is removed, replication will be unable to continue. The default value of binlog retention hours is NULL, which means binary logs aren't retained.

    To specify the number of hours to retain binary logs on a DB instance, use the mysql.rds\_set\_configuration function with a binlog retention period long enough for replication to occur. `24 hours` is the recommended minimum.

    ```text theme={null}
    mysql=> call mysql.rds_set_configuration('binlog retention hours', 24);
    ```
  </Step>
</Steps>

<h2 id="binlog-parameter-group-rds">
  Configure binlog settings in the parameter group
</h2>

The parameter group can be found when you click on your MariaDB instance in the RDS Console, and then navigate to the `Configurations` tab.

<Image img="https://mintcdn.com/private-7c7dfe99-vortex-format/Kj4wT80mHNH7Aczn/images/integrations/data-ingestion/clickpipes/mysql/parameter_group/rds_config.webp?fit=max&auto=format&n=Kj4wT80mHNH7Aczn&q=85&s=55bea055129830191a1fac7e4414fe32" alt="Where to find parameter group in RDS" size="lg" border width="708" height="853" data-path="images/integrations/data-ingestion/clickpipes/mysql/parameter_group/rds_config.webp" />

Upon clicking on the parameter group link, you will be taken to the parameter group link page. You will see an Edit button in the top-right:

<Image img="https://mintcdn.com/private-7c7dfe99-vortex-format/Kj4wT80mHNH7Aczn/images/integrations/data-ingestion/clickpipes/mysql/parameter_group/edit_button.webp?fit=max&auto=format&n=Kj4wT80mHNH7Aczn&q=85&s=12850466cc300e3ed19c43c99bcc72f7" alt="Edit parameter group" size="lg" border width="1662" height="292" data-path="images/integrations/data-ingestion/clickpipes/mysql/parameter_group/edit_button.webp" />

Settings `binlog_format`, `binlog_row_metadata` and `binlog_row_image` need to be set as follows:

1. `binlog_format` to `ROW`.

<Image img="https://mintcdn.com/private-7c7dfe99-vortex-format/Kj4wT80mHNH7Aczn/images/integrations/data-ingestion/clickpipes/mysql/parameter_group/binlog_format.webp?fit=max&auto=format&n=Kj4wT80mHNH7Aczn&q=85&s=bb0f6924a04f3dd670b2a043e1497a8d" alt="Binlog format to ROW" size="lg" border width="960" height="232" data-path="images/integrations/data-ingestion/clickpipes/mysql/parameter_group/binlog_format.webp" />

2. `binlog_row_metadata` to `FULL`

<Image img="https://mintcdn.com/private-7c7dfe99-vortex-format/Kj4wT80mHNH7Aczn/images/integrations/data-ingestion/clickpipes/mysql/parameter_group/binlog_row_metadata.webp?fit=max&auto=format&n=Kj4wT80mHNH7Aczn&q=85&s=96e2a974ea96455d04d46670d205275f" alt="Binlog row metadata to FULL" size="lg" border width="934" height="234" data-path="images/integrations/data-ingestion/clickpipes/mysql/parameter_group/binlog_row_metadata.webp" />

3. `binlog_row_image` to `FULL`

<Image img="https://mintcdn.com/private-7c7dfe99-vortex-format/Kj4wT80mHNH7Aczn/images/integrations/data-ingestion/clickpipes/mysql/parameter_group/binlog_row_image.webp?fit=max&auto=format&n=Kj4wT80mHNH7Aczn&q=85&s=63cfc00fc65e0875ef1a28c0bce08f37" alt="Binlog row image to FULL" size="lg" border width="934" height="234" data-path="images/integrations/data-ingestion/clickpipes/mysql/parameter_group/binlog_row_image.webp" />

Next, click on `Save Changes` in the top-right. You may need to reboot your instance for the changes to take effect. If you see `Pending reboot` next to the parameter group link in the Configurations tab of the RDS instance, this is a good indication that a reboot of your instance is needed.

<br />

<Tip>
  If you have a MariaDB cluster, the above parameters would be found in a [DB Cluster](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.CreatingCluster.html) parameter group and not the DB instance group.
</Tip>

<h2 id="gtid-mode-rds">
  Enabling GTID Mode
</h2>

Global Transaction Identifiers (GTIDs) are unique IDs assigned to each committed transaction in MySQL/MariaDB. They simplify binlog replication and make troubleshooting more straightforward. MariaDB enables GTID mode by default, so no user action is needed to use it.

<h2 id="configure-database-user-rds">
  Configure a database user
</h2>

Connect to your RDS MariaDB instance as an admin user and execute the following commands:

1. Create a dedicated user for ClickPipes:

   ```sql theme={null}
   CREATE USER 'clickpipes_user'@'host' IDENTIFIED BY 'some-password';
   ```

2. Grant schema permissions. The following example shows permissions for the `mysql` database. Repeat these commands for each database and host that you want to replicate:

   ```sql theme={null}
   GRANT SELECT ON `mysql`.* TO 'clickpipes_user'@'host';
   ```

3. Grant replication permissions to the user:

   ```sql theme={null}
   GRANT REPLICATION CLIENT ON *.* TO 'clickpipes_user'@'%';
   GRANT REPLICATION SLAVE ON *.* TO 'clickpipes_user'@'%';
   ```

<h3 id="iam-authentication">
  Using IAM authentication (optional)
</h3>

<Note>
  AWS [introduced IAM database authentication for RDS for MariaDB in version 10.6.5](https://aws.amazon.com/blogs/database/iam-authentication-with-amazon-rds-for-mariadb/). Support depends on the exact engine version and AWS Region. Check [AWS's current supported Regions and engine versions](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RDS_Fea_Regions_DB-eng.Feature.IamDatabaseAuthentication.html) before using this option.
</Note>

<IAMAuthentication engine="mysql" service="rds" />

<h2 id="configure-network-access">
  Configure network access
</h2>

<h3 id="ip-based-access-control">
  IP-based access control
</h3>

If you want to restrict traffic to your RDS instance, please add the [documented static NAT IPs](/integrations/clickpipes/networking/static-ips) to the `Inbound rules` of your RDS security group.

<Image img="https://mintcdn.com/private-7c7dfe99-vortex-format/Kj4wT80mHNH7Aczn/images/integrations/data-ingestion/clickpipes/mysql/source/rds/security-group-in-rds-mysql.webp?fit=max&auto=format&n=Kj4wT80mHNH7Aczn&q=85&s=0e56415714ed3af2e3736dfe22742344" alt="Where to find security group in RDS?" size="lg" border width="2850" height="994" data-path="images/integrations/data-ingestion/clickpipes/mysql/source/rds/security-group-in-rds-mysql.webp" />

<Image img="https://mintcdn.com/private-7c7dfe99-vortex-format/mRaTj__cZTg8Xed9/images/integrations/data-ingestion/clickpipes/postgres/source/rds/edit_inbound_rules.webp?fit=max&auto=format&n=mRaTj__cZTg8Xed9&q=85&s=56fcfa6c33762efe76020d3e5e305038" alt="Edit inbound rules for the above security group" size="lg" border width="1800" height="935" data-path="images/integrations/data-ingestion/clickpipes/postgres/source/rds/edit_inbound_rules.webp" />

<h3 id="private-access-via-aws-privatelink">
  Private access via AWS PrivateLink
</h3>

To connect to your RDS instance through a private network, you can use AWS PrivateLink. Follow our [AWS PrivateLink setup guide for ClickPipes](/resources/support-center/knowledge-base/cloud-services/aws-privatelink-setup-for-clickpipes) to set up the connection.