Looping in automations is here!

[ScottWorld]

I’m not betting against @bfrench — he hasn’t been wrong yet!

[Karlstens]

I’ve finally noticed it pop-up in my production base for one of my clients. Had a little tinker, and taking notes from @Kuovonne’s arguments I could consider the no-code looping useful within that context. I’ll need some time to explore. It might be a case of that time where I found by simply trailing an “Update Record” action after my script, it did actually save me writing a bunch of code within my script to just push outputs into an email action (sounds weird, but I found that the “Update Record” action has this ability to output the latest record info too)

However, I think I’ll continue to use scripts for the bulk of my work, as I really, really enjoy writing JavaScript and solving problems with it.

Lastly, from exploring the updated no-code loop group, if anything, it reminded me about the major limitation within Automation Conditional Groups, where they don’t support nested groups for AND OR conditions (as you find within the Grid View filter). Being forced to use either AND or OR, and not being able to mix them up in sub-groups is a real pain in the …

EDIT: One thing I can’t help but wonder, is that now there’s this new sub-option for either Looping or Conditional Logic, I wonder if they’ll add an area for global scripts - which in theory, could allow any automation to call the one script - which would make for easy maintenance of that segment of code that’s called by multiple automations. … imma gonna feature request.

[AirBenderMarcus]

I’ve been trying to get a list of linked records from one record to link to another in the same order. Even tried to do this with records from a view. The looping automation always links them in a random order each time. I can sort afterward with a script but that defeats the purpose of using the loop feature anyway.

So far it hasn’t improved anything I’m trying to do.

[ScottWorld]

Yeah, unfortunately, many of Airtable’s automation actions (not just the new looping actions) will return records in a random & unpredictable order. What they really need in their automations is a “Sort Records” action.

Here’s what their support article says about this for looping:

Each loop iteration is not guaranteed to run in a certain order. If your automation absolutely requires that there be a predefined order to the way that the list of records is looped through, then you’ll likely need to explore other options.

Although they don’t mention what they mean by “other options”.

What they really mean is “scripting” or using an external automation tool like Make.

[ScottWorld]

p.s. I just added this to the feature request post here:

List your top Airtable feature requests here!

Another feature request: Sorting records in automations: Many of Airtable’s automation actions, including the new looping actions, will return records in a random & unpredictable order. What Airtable really needs in their automations is a “Sort Records” action.

[ScottWorld]

Here’s a nice thread on how you can use repeating groups (i.e. looping) in automations to create personalized emails for unique individuals on a schedule:

Single automation to send a customized daily digest to different emails

I have a table of content that gets updated to a new status and I track the date of that change in a field. This table has content for many different clients (client is a field), and I want to send each client a daily digest of the items that moved into that particular stage. A digest email for a single company is not hard, but I’m wondering if I can, in a single automation, send a daily digest of the updated content for each company to that company’s contact (their email address is also a field…

[Databaser]

Am I missing something? I can’t limit a “grid” or “list” from a “find records” action to the different values of the used “input list”?

eg: find records where “status” = to do; loop based on “owner”; send grid of records that are only tied to that owner? In the “current item” option, I can’t add a “list” or “grid”? So it does email everyone separately, but showing every record from the “find records” action.

[ScottWorld]

It’s pretty unintuitive. You would have to do yet another “find records” step within the repeating/looping section of your automation that performs a find for the specific records you’re looking for.

(Unlike Make.com’s automations, this won’t create another loop… it just gives you the chance to grab a list of the records you want.)

Back in your database, you’ll probably need to create formula fields and lookup fields (possibly resulting in the Record ID) that let you do the proper searches in your automations.

[bfrench]

@ScottWorld

You would have to do yet another “find records” step within the repeating/looping section of your automation

And there lay the key design flaw in this new looping feature. The pervasive use of this feature will serve convenience well and disserve performance.

Airtable engineers should have tackled this challenge with a collection of automation components - one is a grid extraction (aka, a hash index) that would serve as a blistering fast approach to mitigate nested looping altogether.

[bfrench]

@ScottWorld

this won’t create another loop… it just gives you the chance to grab a list of the records you want.

Note to self: I just liked a Make feature. Book an extra session with my shrink.

[Kuovonne]

@bfrench

And there lay the key design flaw in this new looping feature. The pervasive use of this feature will serve convenience well and disserve performance.

Currently Airtable automations can have only one no-code loop. Thus a no-code automation is limited to one level of loop nesting. I wonder if that is part of their attempt to manage performance issues.

[bfrench]

@Kuovonne

I wonder if that is part of their attempt to manage performance issues.

I think it is. But more accurate to say:

A nested loop is a really poor design pattern; there shouldn’t be any nesting at all. You and I have proven this many times in script.

[Kuovonne]

@bfrench

A nested loop is a really poor design pattern

A nested loop is one of several design patterns and it is easier for many emerging developers to understand than the more efficient design patterns. As such, I think it has a place in the world.

But this is also where I think Make has a better pricing strategy. Make charges per operation so more complex scenario cost the user more. If the user chooses to use an inefficient design pattern, they pay the price for it.

On the other hand, Airtable only charges a flat rate per automation trigger. If I’m reading the documentation correctly, you could then run an automation that runs a complex 30 second script over 150,000 times for the cost of a single automation run. I’m probably reading the documentation wrong.

[bfrench]

@Kuovonne

an automation that runs a complex 30 second script over 150,000 times for the cost of a single automation run. I’m probably reading the documentation wrong.

I think you’re interpreting the docs just fine. But, a variable (perhaps inefficient) process that must run in the constraint of 30 seconds is difficult to predict. Airtable scale, this approach will almost certainly break, right?

[Kuovonne]

@bfrench

I think you’re interpreting the docs just fine.

Thanks.

@bfrench

this approach will almost certainly break, right?

I’m pretty sure the automation that I described would fail. It is an extreme case to illustrate a point, not a reasonable automation. But I don’t know what the actual limit is or should be. And I wonder what will happen when people bump up against undocumented but realistic limits.

[bfrench]

@Kuovonne

I wonder what will happen when people bump up against undocumented but realistic limits.

Confusion. Dismay. Support calls.

[RussellBishop]

Just to add to this, I think you would probably initially do a find on your Users, then loop on those Users, and do a find records for their Tasks (status = to do") within your loop.

[bfrench]

@RussellBishop

then loop on those Users, and do a find records for their Tasks (status = to do") within your loop.

But, circling back to the issue - nested loops (without the benefit of a hash index) are performance hogs. If the data sets are small, this works just dandy. But as the task data grows, the performance will always be in a state of increasing decline, and such decline is non-linear. It won’t scale.

You need a hash index of tasks so that they are traversed one time and all lookups are by memory pointers (i.e., user IDs that reveal in < 10ms all the tasks for that user).

[Kuovonne]

@bfrench

circling back to the issue - nested loops (without the benefit of a hash index) are performance hogs.

I agree that a hash index is better. But what is a no-code Airtable creator realistically able to do about this?

I think this is an issue for Airtable to solve. Or perhaps they have already decided to take the performance hit (at least for now).

[bfrench]

@Kuovonne

Or perhaps they have already decided to take the performance hit (at least for now).

Unfortunately, it is not Airtable who takes the hit; it is users who feel this. Airtable’s own servers are insulated from their poorly-designed features.