> ## Documentation Index
> Fetch the complete documentation index at: https://domoinc-openapi-sync-dataflows.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Magic ETL Tiles: Filter

export const InlineImage = ({src, alt = '', height = '1.6em'}) => {
  return <img noZoom src={src} alt={alt} style={{
    display: 'inline',
    verticalAlign: 'start',
    height: height,
    margin: '0'
  }} />;
};

### Intro

This article describes the available Filter tiles in Magic ETL: Filter Rows, Split Filter (limited release), and Remove Duplicates.

<Frame>
  <img alt="Filter_Tab.png" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/ag1wPXP26szkgB9J/images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EM5w000005vNiv.png?fit=max&auto=format&n=ag1wPXP26szkgB9J&q=85&s=99ee45d34397b027432e1144f55ae2db" width="273" height="159" data-path="images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EM5w000005vNiv.png" />
</Frame>

Learn to [create a Magic ETL DataFlow](/s/article/360055259234 "Creating a Magic ETL DataFlow").

***

### Filter Rows Tile

<Frame>
  <img alt="Filter_Rows.png" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/ag1wPXP26szkgB9J/images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EM5w000005vNit.png?fit=max&auto=format&n=ag1wPXP26szkgB9J&q=85&s=a4704a903964ab30fd1e4c9f10137dc8" width="90" height="98" data-path="images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EM5w000005vNit.png" />
</Frame>

The Filter Rows tile lets you include or exclude rows based on specified rules.

**Example**

For example, the Filter Rows tile transforms the column in the first image to the column in the second image.

<Columns>
  <Frame>
    <img src="https://mintcdn.com/domoinc-openapi-sync-dataflows/YY6W1U347GpqLSJS/images/kb/ka05w00000124TD-00N5w00000Ri7BU-0EM5w000005vNfw.png?fit=max&auto=format&n=YY6W1U347GpqLSJS&q=85&s=b164d9574e13c8e10ea27ed130b28a2e" width="599" height="388" data-path="images/kb/ka05w00000124TD-00N5w00000Ri7BU-0EM5w000005vNfw.png" />
  </Frame>

  <Frame>
    <img alt="etl_filterrows_dataout.png" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/YY6W1U347GpqLSJS/images/kb/ka05w00000124TD-00N5w00000Ri7BU-0EM5w000005vNfq.png?fit=max&auto=format&n=YY6W1U347GpqLSJS&q=85&s=352bdaebc5d562189ba6376b23e167fb" style={{width: 400, height: 259}} width="599" height="388" data-path="images/kb/ka05w00000124TD-00N5w00000Ri7BU-0EM5w000005vNfq.png" />
  </Frame>
</Columns>

The configuration for this transformation is as follows:

1. Choose the `Product Container` column to filter on.
2. Choose a `contains` comparison.
3. Choose to compare against a specific value.
4. Enter the value `Large Box.`

<Frame>
  <img alt="etl_filterrows_sampleui.png" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/YY6W1U347GpqLSJS/images/kb/ka05w00000124TD-00N5w00000Ri7BU-0EM5w000005vNgP.png?fit=max&auto=format&n=YY6W1U347GpqLSJS&q=85&s=70558179fc81862e108962f969f1f8d1" width="710" height="304" data-path="images/kb/ka05w00000124TD-00N5w00000Ri7BU-0EM5w000005vNgP.png" />
</Frame>

You can configure the Filter Rows tile by following these steps:

1. Drag the **Filter Rows** tile from the **Filters** section in the left panel to the canvas and connect it to the rest of your DataFlow.

2. (Optional) Rename the tile.

3. Select either **Add Filter Rule** or **Add Formula Rule**.

   1. If you choose **Add** **Filter Rule**, do the following:

      1. Select a column to filter on.
      2. Select the operation to use.

         Operation items appear in the list, depending on the type of data in the column to filter on.
      3. Select whether to compare against values in a specific column or a specific value, then do one of the following:

         * (Conditional) If comparing against a column, select the column to use.
         * (Conditional) If comparing against a specific value, enter the value to use.
      4. (Conditional) If you want to add another rule, select either **Add Filter Rule** or **Add Formula Rule**.
   2. If you choose **Add** **Formula Rule**, do the following:

      1. Enter the formula you want to use.
      2. Select
         <InlineImage src="/images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000008iTw9.jpg" /> **Validate** to check the formula.

      OR

      1. Select **Open the Formula Editor**.

         <Frame>
           <img alt="" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/ag1wPXP26szkgB9J/images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000008iTuX.jpg?fit=max&auto=format&n=ag1wPXP26szkgB9J&q=85&s=015a198fb7ad6c1b9d3a65b630328e9e" style={{width: 874, height: 148}} width="2812" height="476" data-path="images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000008iTuX.jpg" />
         </Frame>

      2. In the expanded formula editor, enter the formula you want to use in the **Calculation** field.

         <Note>
           **Use the AI SQL Assistant:** The AI SQL Assistant is available in the expanded formula editor. Use the prompt field at the bottom of the **Calculation** field to enter your query in natural language. The SQL appears above in the **Calculation** field.

           <Frame>
             <img alt="" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/ag1wPXP26szkgB9J/images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000008iUQn.jpg?fit=max&auto=format&n=ag1wPXP26szkgB9J&q=85&s=ef26aac0fb6adbdb71cfc47c2a264523" style={{width: 749, height: 190}} width="2908" height="738" data-path="images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000008iUQn.jpg" />
           </Frame>
         </Note>

      3. Use the **Validate** option to check the formula.

      4. Save and close to return to the canvas.

4. To add another rule, choose either **Add Filter Rule** or **Add Formula Rule**.

   <Note>
     **Note:** For more information on creating a successful filter using the Formula Rule, see [Writing a Filter Formula in Magic ETL](/s/article/360044296573 "How to Write a Filter Formula in Magic ETL v2 (Beta)").
   </Note>

5. Select whether to include rows that meet all or any of the rules you define.

<Tip>
  **Tip:** You can preview the data transformed by a tile by running a preview, clicking the tile in the canvas, then clicking the **Preview** tab.
</Tip>

### Split Filter Tile (Limited Release)

<Warning>
  **Important:** The Split Filter tile is currently only available for Magic ETL on Snowflake or Databricks. It is targeted for general release by the end of 2025.
</Warning>

If you have a Domo integration with Snowflake or Databricks, you can access the Split Filter tile in Magic ETL. This tile allows you to have multiple output streams from a filter in your DataFlow. It is useful when you want multiple streams, one to represent each filter condition, and/or a stream to represent everything that is dropping out of the DataFlow because of the filter.

# Configure Split Filter Tile

1. Drag a Split Filter tile to the canvas from the **Filter** section of the left panel. This tile has a bigger footprint on the canvas than a standard tile, and it grows as you add more output streams.
2. Connect the tile to your DataFlow.

   <Frame>
     <img alt="" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/ag1wPXP26szkgB9J/images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000008k6KL.jpg?fit=max&auto=format&n=ag1wPXP26szkgB9J&q=85&s=240cf457879c93d1dc2564df0bbd8972" style={{width: 549, height: 298}} width="842" height="458" data-path="images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000008k6KL.jpg" />
   </Frame>
3. Configure the formula filters. Each formula represents a different stream that will be output from the tile.

<Note>
  **Note:** In this tile, you can only configure formula filters and not filter rules.
</Note>

4. Choose whether or not to output all unfiltered rows as an "else."
5. (Optional) Name any of the filter outputs, including the "else" output.

   The outputs you choose display on the canvas as multiple nodes.

   <Frame>
     <img alt="" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/ag1wPXP26szkgB9J/images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000008k4s2.jpg?fit=max&auto=format&n=ag1wPXP26szkgB9J&q=85&s=4206bd7ffa02fc639318285cbbdd5330" style={{width: 700, height: 232}} width="1227" height="407" data-path="images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000008k4s2.jpg" />
   </Frame>
6. Build your downstream DataFlow from any of the outputs.

### Remove Duplicates Tile

<Frame>
  <img alt="Remove_Duplicates.png" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/ag1wPXP26szkgB9J/images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EM5w000005vNio.png?fit=max&auto=format&n=ag1wPXP26szkgB9J&q=85&s=d2e9d045eac7ab404a8d1cb5cccbcaee" width="85" height="113" data-path="images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EM5w000005vNio.png" />
</Frame>

The Remove Duplicates tile lets you remove duplicate rows of data based on specific columns.

Follow these steps to configure the Remove Duplicates tile when you have rows with identical data:

1. Bring your DataSet into Magic ETL as an input.
2. Drag a **Remove Duplicates** tile to the canvas from the left panel under **Filter**.
3. Connect your data to the tile.
4. Select the input DataSet tile to open the tile editor below the canvas. In step 1 of the **Configuration** tab, choose the columns to use to check for duplicates. Choosing one or more columns that correctly identify a unique row keeps you from removing vital data. For example, if there is a column of `Transaction IDs` that are unique for each sale in the data, using this column ensures that you only remove true duplicates.

   A row is only removed if *all* of the columns selected in the Remove Duplicates tile are duplicates.

   <Note>
     **Note:** By default, all columns are selected. You can remove columns one at a time or use the **Remove All Columns** option. To re-add a column, you can choose **Add Column** to add individual columns or use the **Add All Columns** option.
   </Note>

   <Frame>
     <img alt="Screenshot 2024-07-30 at 12.58.24 PM.png" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/ag1wPXP26szkgB9J/images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000001zgTC.jpg?fit=max&auto=format&n=ag1wPXP26szkgB9J&q=85&s=6b9b810bcc872b74beccabad33e27eb2" style={{width: 700, height: 319}} width="1516" height="692" data-path="images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000001zgTC.jpg" />
   </Frame>
5. (Optional) By default, key columns are case-sensitive. Using the checkboxes, you can individually make columns case-insensitive.
6. (Optional) Select **Run Preview** to preview your completed data transformation.

   <Frame>
     <img alt="run preview.jpg" src="https://mintcdn.com/domoinc-openapi-sync-dataflows/ag1wPXP26szkgB9J/images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000001zsSk.jpg?fit=max&auto=format&n=ag1wPXP26szkgB9J&q=85&s=d2abd2151bd7088e57567cddad259e5e" style={{width: 749, height: 299}} width="2826" height="1128" data-path="images/kb/ka0Vq000000G7bp-00N5w00000Ri7BU-0EMVq000001zsSk.jpg" />
   </Frame>
