Documentation

Welcome to Retool! We're a fast way to build custom internal software.

You'll find the 5 minute demo, quickstart guide, and documentation for each of our connectors and components here. If you've got any questions -- chat with us on the bottom right!

Get Started    Guides

Tag images for machine learning

Draw bounding boxes around images, and save them to the database.

Let's say we're a self-driving car company, and need to label a ton of images. We'll build a tool where we take a list of image URLs, let the end-user draw bounding boxes around each image, and save the results back to an API, all in 5 minutes.


If you haven't already, you can create a Retool account to follow along.
Create an account

1. Get list of images

Let's first switch to the onboarding_db (readonly), since we provide you some pre-loaded data.

Then, write a SQL query to pull in all of our images:

select * from images order by id;

Let's preview it. If that looks good, let's save it. Our query editor should look like this right now:

A query to pull in all our `Image`s.

A query to pull in all our Images.

Great! Now let's drag on a Table from the right hand side. It should be automatically populated with data from your last query (in this case, the images). Great - that's it! Images ready to be labelled, in our Table. Your app should currently look like this:

An `Table` with all our `Image`s.

An Table with all our Images.

2. Render and label them

Now let's render each image onto a tagging canvas. Let's drag in a BoundingBox component from the right hand side. It's near the bottom, so it's easier to use the search box on top:

Searching for the `BoundingBox` component on the right hand side.

Searching for the BoundingBox component on the right hand side.

Let's drag it on, and then change the image_url to refer to what's currently selected in the table. That'd be {{table1.selectedRow.data.image_url}}.

Great! Now your BoundingBox should be showing the selected image in the Table. And when you select different rows in the Table, the BoundingBox will show a different image for tagging.

Even though our data currently has no bounding boxes, we probably want to show the bounding boxes - if there are any in the database. See the "Initial Bounding Boxes" on the right hand side? Let's change that to: {{JSON.parse(table1.selectedRow.data.labels)}}. That currently won't do anything, since we don't have any bounding boxes, but this'll come into use later.

A working `BoundingBox` component. When the selected row in the `Table` changes, the image in the `BoundingBox` changes too.

A working BoundingBox component. When the selected row in the Table changes, the image in the BoundingBox changes too.

Let's try labeling a few boxes before moving on to the next step. The BoundingBox supports panning and zooming when you hold down the Command key (on Macs), and Control key (on Windows / Linux).

3. Save labels

Cool! Now we just have to save the tags to our database. Let's drag on a Button, and label it "save tags". Let's also make sure it's action is run a query, and that we create a new query:

Creating a new query to save our labels.

Creating a new query to save our labels.

Let's change the created query's resource to onboarding_api, since we want to PUT data back. Then fill in the values below to construct the PUT query. We're going to be PUTting back to the selected image, whose id is {{table1.selectedRow.data.id}}.

For the body, the key will be labels, and the value will be:

{{ JSON.stringify(boundingbox1.boundingBoxes) }}
Filling out our `PUT`.

Filling out our PUT.

And after the query is successfully ran, we want to refresh our original query, so we can see the new bounding boxes. So let's scroll down and change "on success, trigger these queries" to include our original query (query1):

Refreshing `query1` after our `PUT` is successful.

Refreshing query1 after our PUT is successful.

That's it! Let's save the query and try it out. Draw some bounding boxes, and press the Button. Your bounding boxes should be saved to the database, and your Table will refresh. Great - bounding boxes there!

Since we set our BoundingBox to render boxes if they exist, you can click around to a different row in the Table. When you come back, you'll see that the bounding boxes are still there. So you can add some boxes, and then press save again - all without losing data.

4. Actually using it

This tool is now production-ready. You can send the link to your coworkers, and they'll be able to use it to tag data - immediately. If you'd like, you can also build out an approval workflow (so managers have to approve labelled images), modify the SQL query to only pull in images that are unlabelled / assigned to the particular user (try the current_user variable), etc.

If you give your operators specific permissions, they won't be able to edit the tool and change the queries - they'll just be able to use the tool you've created.

Good job! A tool for tagging images - all in 5 minutes.


Want to build your own tools for ML?
Create an account


-->