Journey from Chaos to Control — Introduction to Automation

Over the years we switched most of our projects to autopilot, thanks to automation. Now we discovered, that with AI and Alice we can go even further! Here are the fundamentals of this approach.

Below you will find some essential concepts of automation, as we use it. Even though some more technical people might find it pretty basic, we feel this it’s necessary that all of us are on the same page. In future, we’ll provide some more in-depth guides, in the meantime, I also recommend some previous articles:

Rethinking productivity with AI & Alice
***
Rethinking ChatGPT on macOS — Alice 2.0 UI reveal
***
Plain text is what you need. But why?

Today I slept through a disaster.

At 4:35 a.m. I got the text message that one of my servers is down. Reason? Trivial. No funds on a connected credit card. Did I miss some notifications beforehand? No. I turn them all OFF by default. I consciously opt-out of any things that may distract my work and life. And with over 150 subscription services that my business handles I’d be notified every millisecond. But hey! Shouldn’t notifications that come before the blackout be AT LEAST the ones that I opt-in for? No.

I have robots to deal with them 🤖

At 4:35 one of them got woken up. He checked the balance on my different disposable credit cards, took the first one with sufficient funds and plugged it in via hosting service billing API. At 4:36 I got another text message. Server Status: up.

I spotted the messages after I went cycling, had a shower and morning coffee, around 8. Good ol’ bots — I thought.

The first time the idea resembled me while reading about nomadlist ran by Pieter Levels, who hired over 1000 bots on his site, including those calculating beer prices 🍺 I picked up the idea and started learning about cron, automation, AWS Lambda, web scrapers and writing first small pieces of software.

Holy sh*t, that’s awesome! — I thought.

I’ve learned to code early in my career, but then switched to web design, UX and biz dev. Although I could code anything I didn’t find it too exciting. It was taking too long and I kind of missed the instant gratification you get from graphic and motion design.

When I discovered bots and automation it instantly felt like the missing piece of the puzzle in my life just got in place.

The results are quick and powerful and no or little code is required to make software, that actually works.

As I was exploring more tools and techniques I discovered that almost every daunting part of my work can be replaced with smart automation. Starting from organising my schedule and sending invoices to my accountant, to running an entire online business on autopilot, with bots taking care of purchases and returns, invoicing, customer service and marketing. Result? Today I run multiple online projects including the one with 300k clients and only few people on board, thanks to bots and automation.

Automation as I know it

Most of the people I talk to have come across the concept of automation. However, their understanding of this topic is different from what I want to present, so I believe it is very important for you to be aware of how the evolution of automation looks from my point of view and how I understand this concept. For me, automation is the use of available tools to minimize human involvement in various areas of online projects, such as sales, customer service, accounting, or marketing. I want to share my approach and set of techniques that I use every day to make my work simpler and not involve me where it is not necessary.

A simple example is handling returns where we need to issue a corrected invoice, receive confirmation from the customer, process the return, and book it. Seemingly a simple operation, as it turns out, in larger companies, it involves several positions. Polish airlines LOT recently processed my refund for a canceled flight for 7 months. That's not all; changing company data on an invoice in the Microsoft Azure service took almost a year and required the exchange of 151 emails. Meanwhile, while the thread was open, one of the customer service representatives managed to get pregnant, give birth, and go on maternity leave. In Vimeo, a similar process is still ongoing, I can't count the messages, but at the moment, 14 people on the Vimeo side are involved, including 4 managers, legal department personnel, accountants, and several others.

You could say these are procedures of large corporations. Meanwhile, in smaller companies, even though the process may be faster, most of the time nothing is described, and each case is considered individually, with the peak of dreams being template responses that slightly speed up the work.

Meanwhile, all repetitive tasks can be delegated to robots and automated, almost entirely, even if the process is complex and requires confirmation exchanges. On one hand, this is a huge time and money saver for companies where hours are spent on boring and repetitive tasks, and on the other hand, a huge opportunity to gain a competitive advantage, or even compete with much larger companies, for smaller ones. Another huge benefit of automating boring, monotonous tasks is unleashing the potential of people who could be much more creative and satisfied with their work.

For years, there was a belief (and few still think differently) that automation requires specialized knowledge and armies of programmers. Indeed, until now, process optimization was mainly done by larger companies that employed RPA (Robotic Process Automation) specialists, people who first spent 3 months recognizing a process, then produced dedicated software for several months, and then maintained it indefinitely. You could say that automation was only available to the largest and wealthiest.

Probably the most revolutionary change in this area is the evolution of tools that support automation and, consequently, the democratization of these techniques. Today, literally everyone is able to automate, and most importantly, reaching a truly advanced level does not require any programming knowledge. Additionally, tools supporting automation are affordable, which also guarantees wide access to technology.

The background (boring but worth it)

When the Internet became almost universally accessible, options for data synchronization in the cloud and the ability to collaborate simultaneously on the same data emerged. Initially, there were project management applications and then document editing tools (Google Apps). Today, even designers can work simultaneously on the same project, for example, using Figma. This clearly shows a progression, leading us from a situation where applications worked independently to the point where they moved to the Internet and browser-based versions. This facilitated collaborative work and access to our data from any device.

And since we're talking about devices, we have gone through a revolution related to mobile devices, or rather, multi-platform support. This has made the process of software development and design much more complex. Now, developers not only had to develop the main version of the application but also take care of their mobile versions for smartphones and tablets. This led to the strategy of designing application architecture using APIs, which stands for Application Programming Interface.

Definition of API

To understand what an API is, it is important to know that an "interface" is a way to enable communication. And an API is a way to enable communication within one or multiple applications.

This strategy simplified the process of creating applications for multiple platforms because developers no longer had to create multiple versions of the application. Instead (using a simplified schema), a single server providing user data was sufficient for applications created on specific platforms, with a lower level of complexity in this case.

Furthermore, not only the developers of a specific application have access to a well-prepared API but also anyone who obtains access keys (e.g., API key).

https://www.techtarget.com/searchapparchitecture/definition/application-program-interface-API

API-First Design and the Online Business

All of this led to the creation of a software development strategy called "API First". From what I have just written, this approach is fully justified. If we can write one code to handle multiple platforms, why wouldn't we do it?

And what does this mean for non-programmers? Well, the API-First approach can be used as a business strategy and for utilizing applications - thanks to automation.

As we have already mentioned, APIs allow for data exchange within applications and between applications. This opens up entirely new possibilities. It turns out that we can connect various services we use and exchange data between them. This means we are no longer limited by the functions available in selected applications but have a full range of possibilities provided by all the tools we use!

How is this possible? Well, developers provide access to their API not only to their own applications (for example, the mobile and browser application of your bank), but also... to other developers, who can build solutions based on their software.

These other developers will be us, using tools specialized in specific tasks - sending emails (e.g., Mailchimp), SMS messages (e.g., Twilio), enriching customer data (e.g., Clearbit), notifying the sales department (e.g., Slack), etc., and combining them through Automation. With this approach, we will never have to log into the graphical interface of the applications we use, for example, to send marketing emails. Automation will do it for us by "logging in" to the API and sending emails of specific content to a designated group of recipients. This means that neither we nor our employees will ever have to log in (and forget passwords) to the online services we use, which have good, accessible APIs.

The benefits of this approach include:

• No need to use GUI (reloading pages, slow tool operation, forgetting passwords, logging in again - all of these waste time!)

• The ability to connect the best, specialized services instead of relying on one provider who may constantly lack certain functionality

• Reactive work, not proactive work - we don't have to check what needs to be done; automation assigns tasks that humans should complete in the process (more on that later)

Looking at this through the lens of my earlier definition of automation, a technical aspect can be added to the list:

1. Detailed process mapping

2. Attempt to simplify/improve the process

3. Choosing the right tools for the right tasks in the process

4. Connecting these tools through APIs to work at the right time, without the need for human intervention and logging into GUI

5. Supervision, error correction, and experiments

Example of API + Automation

Let's take the example of organizing an online event, such as a webinar. The webinar tool allows for participant registration, so we have access to email addresses and phone numbers. With the help of API and automation, we can automatically transfer this user's data to the Clearbit service responsible for Data Enrichment, which enriches data. By sending, for example, a corporate email address, we receive additional information about the person signing up for our event. These enriched data can also be automatically sent to our CRM system (e.g., Pipedrive), positively impacting the sales process because when contacting such a potential customer, we have more information about them to better tailor our services to their needs. Alternatively, we can use the phone number to automatically send a short SMS reminder about the event. This increases attendance, which can benefit us and our potential customers.

If we combine this with a system that collects customer data on our website, such as Intercom, our salesperson can also receive precise notifications with information like:

Adam from the Browser company logged in 10 minutes ago, he participated in the "Financial Markets" Webinar and visited the pages: offer, faq, contact. His phone number is: 888999000

As you can easily imagine, using one tool in this process is impossible. There is no single tool that can handle webinars, website data, company communicators, etc. And this is the power of automation - we use various tools specialized in specific tasks, juggle their data, and the results go exactly where our team feels comfortable.

So, as you can see, we used the API here to transfer data between different services and thus optimize a process that was either previously done manually or not done at all. This approach streamlines existing processes or creates new ones. I have personally found that in the case of online businesses, this approach truly takes our projects not just to the next level but often ten levels higher.

For example, the projects we have undertaken, considering the scale of operations and their profitability, can easily be compared to businesses that require the involvement of even a team of dozens of people!

Let’s have fun (you survived the boring part)

As we established above, modern applications are increasingly communicating through APIs. Let's take Facebook as an example. The business logic is responsible for adding your post to a certain type of database and your feed. This logic can be triggered from various places. You can add a post through a mobile application or the Facebook website, and as you know well, other applications can connect with Facebook and post on your behalf.

To enforce this logic, you need to:

1. Know where the back-end expects this data (an endpoint URL)

2. Send the data in a way understandable to the back-end (usually in JSON format)

3. Authenticate using certain access data (login, token, etc.)

And voila! The post is on your feed.

How to Utilize the API First Approach

The entire internet works this way. And thanks to this model, everyone can add, modify, and execute commands in web applications.

But instead of becoming a developer and learning how to structure, format, and send data through various APIs, you can use tools that will do it for you. I call them LEGO tools. When you authorize any application in such tools, they can perform various actions on your behalf. For example, create a spreadsheet and add rows to it.

Now, let's think about this more broadly. If LEGO tools can connect with different applications, they can also exchange data between those applications. For instance, these tools can take rows from a spreadsheet and post them on your Facebook feed. Sequencing, timing, delays, conditions - everything can be set.

Infinite Possibilities

Now imagine you want to gather data from your spreadsheet and distribute it to all your social media profiles - Facebook, LinkedIn, Instagram, YouTube, all at once. LEGO tools allow you to connect with numerous applications and services you use and exchange information between them or work on changes in one tool while notifying others.

Here are a few example scenarios:

• Searching for events in your calendar and sending you reminders about the most important ones;

• Collecting tasks you enter into Trello and syncing them with your Google calendar;

• Sending emails to webinar participants, posting on social media, and assigning tasks to your team on Slack - all simultaneously.

The beauty of LEGO tools is that they expand the capabilities of the software you already use, allowing you to achieve practically anything you can imagine. Often, the tool you are using may lack just one feature, such as text reminders. With LEGO tools, you can add a text messaging service to your existing arsenal of options in just a few minutes.

Popular LEGO Tools

The most popular LEGO tools are zapier and Make. Its main advantage is that it supports hundreds of available applications. It offers a clear and intuitive interface for creating automations. Make It has a very interesting, visual interface that allows you to build automations by literally connecting a few dots. Additionally, it offers a rich, free plan to experiment with automation.

These are the tools I use most often, but there are also many other tools, usually designed for more specific tasks.

• Parabola - a very good tool for data transformation. This means reorganizing, splitting, changing the order, formatting, and much more. If you have data from multiple sources or a lot of data that you want to transform and send to other tools, this is a great choice.

• Actiondesk - based on a spreadsheet interface, allowing manipulation, transformation, and creating logic within your data. If you like spreadsheets, it's worth checking out this tool. It allows you to make various transformations on data and send it to another tool.

• Retool - this tool fetches data from APIs or databases and allows you to transform it or even send commands back for execution. If you want to create an admin interface for your database or a report panel, this is a good option.

Additionally, there are other tools, created for specific actions, that I don't categorize as LEGO tools - they allow you to combine one or two services and create something new. A good example is Glide, which lets you create a mobile app from a spreadsheet. There are many interesting tools of this type, but that's a topic for another article.

If you want to learn more about different tools and techniques we use for automation and productivity, make sure you check out Everyday on Autopilot — our productivity and automation bible, currently 50% off for Techsistence readers only 💥 This is the best start to your automation journey. Here’s few recent reviews from Product Hunt.

I think one of top books in it's category. Shows interesting approach to automation. I get to know bunch of new tools from it.
— Ben

Imho this ebook and other information products from this authors are the best quality and most advanded soruce of knowlege about proces automation and productivity improvment! Shut up and take my money again! ;)

— Jakub

Mind-blowing knowledge. There's no fluff only apply-at-once solutions. Very reliable. Worth every minute and money spent. I thought I knew something about automation and was enjoying riding a bike instead of walking... these guys showed me Floo Powder!

— Karolina

Your first automations!

All of this may seem overwhelming. But once you understand the concept behind LEGO tools, it will be quite easy to implement and learn any new tools. It's worth knowing at least a few of them. Sometimes the application you want to automate is available in Zapier but not in Make. Sometimes you want to work with a larger amount of data, and Parabola may be more convenient. Unfortunately, there isn't a single LEGO tool that meets all your expectations, but that's... not a bad thing. By learning different tools, you encounter diverse scenarios and thought patterns.

When I fully grasped how Zapier works, I was very excited. But then I struggled with implementing this concept into my business. Tutorials were available but taught everything from the wrong angle, saying "connect X to Y" instead of "achieve X by doing Y." This made it very difficult for me to work with these tools, and it took me over a year to come up with good automation strategies and their utilization.

Automation in Zapier

First automation we will prepare based on one of the most popular platforms in the world - Zapier. This tool is very beginner-friendly and allows you to connect various tools and transfer data between them without coding. Our first automation will show you how to work with Zaps, as Zapier calls scenarios, or individual "robots".

What will we automate? We will create a simple automation that will save your Gmail attachments to a Google Drive folder. Then we will slightly modify it to have a practical application by saving the invoices you receive in your mailbox in one place.

1. Start by creating a free account on Zapier. To do this, go to zapier and register, then create a new Zap by clicking this button, and then Open Editor, creating a Custom Zap.

   

2. Now you need to add a trigger - something that will start your robot. Search for and select the Gmail application. There’s an experimental AI search above, but we’ll skip it to better understand the process

   

3. In the next step, you need to connect your Google account with the Zapier application and specify the appropriate mailbox. You only do this once, and subsequent automations will not require reconnection

   

4. The trigger event for us is New Email Matching Search, meaning we want to trigger the robot every time we receive a message with specific topic and attachment

   

5. Next, we can fine-tune our trigger, and here Zapier offers a very interesting possibility. We can perform more complex queries, according to the Gmail search documentation. For example, I used invoice OR bill AND filename:pdf, which means among messages with attachments, it will only search for those containing the phrase invoice or faktura (or similar because I didn't use quotation marks). Additionally, I am only looking among results containing PDF as an attachment

   

6. You can test your automation with the Test Trigger feature, and if the mailbox already contains elements that match the entered phrase, you should receive them as results (if there is nothing, simulate test data by sending yourself an invoice to the mailbox)

   

7. And you should be able to see results

   

   Select the appropriate result and click Continue.

8. Great! We have configured our trigger, meaning we can receive messages with attachments, additionally filtered. In this case, we only need to configure what will allow us to save the attachment to Google Drive. As part of the Action (what happens as a result of the trigger's action, we search for Google Drive)

   

9. Then, we choose what we want to do, which is to upload the file to Google Drive

   

10. Connect your Google Drive account, or select it if it has been previously connected

11. If you haven't created the appropriate invoice folder on Drive before, now go to drive.google.com and create a folder named invoices. If it's already created, you will be able to specify it as the location in Zapier

    

12. Now click on the file and from the drop-down list, select the Attachment option, simply passing the attachment as a file to Drive. Leave the other fields unchanged, except for the File Name field, where you can get creative and change the name of the saved file, using various metadata from the email, such as the sender's address, combining it with the file name. I added hyphens between them for clarity

    

    Tip: If you paste the following phrase in the Zapier field: {{zap_meta_human_now}}, Zapier will substitute the current date in that place. By formatting names in this way, it will be easy to find files with the correct date. Additionally, sorting by name or date will now return neatly arranged results, which means you don't need to create additional folders (but if you want to do it, feel free to do so. Try modifying the automation to place invoices in a folder with the correct name, for example, January. You will need to create these folders manually or with the help of another automation).

13. That's it! Select Continue, and then Test & Continue to check the automation's operation. If everything went smoothly, you will see a success screen

    

    Now you can Publish your scenario! Test invoice is already on Google Drive

    

    Remember to turn on the Zap using the switch at the bottom so that it starts analyzing all incoming messages from now on. Otherwise, the Zap won't work!

Automation with Make

We prepared our first automation in Zapier for a reason. Make, the application we will be working with now, is not as good at integrating with Google tools. You could say that Zapier is better aligned with Google, and this is a good example of why it's worth having various automation solutions and using the ones that work best in a given situation. In Make, to use Gmail, you need to have a paid Google Workspace account.

Although automating with Zapier is incredibly simple and intuitive, there are many other tools in the world of automation that you will use. Why? There are many reasons. For example, Zapier is a bit expensive, and you can find cheaper alternatives. Additionally, while it offers many interesting features, other tools sometimes work better and function slightly differently. In Zapier, although it is possible, splitting scenarios based on conditions is not the most intuitive.

For example, we can create a bit more advanced scenario in Make, that searches for invoices then renames, catalogs and save them to drive, checking if the folder structure is correct. It looks like this:

On the left, we have the trigger, which listens to emails on Gmail. Then we rename the file, and search for the appropriate folder on drive. The folder should contain month and year to catalog our invoices, so if it doesn’t exist, we will create it in the next step and then put the invoice there. You don't need to understand exactly what these elements mean right now, but the concept should be a bit clearer now.

What will we automate this time? We won't repeat the same task in Make. Instead, we will add another automation that will send a notification on the 10th day of the month, informing our accountant or our email address that the collected documents are ready.

1. Create an account on Make (this is our special link, that grants you 1 free month on the pro plan 🎉)

2. Create a new scenario (equivalent to a Zap) using the Create Scenario option

   

3. Click on the first circle and select the Gmail application

   

4. Choose the action Send an email

   

5. Click Add in the Connection option, name your connection, and like in Zapier, connect to the appropriate Google account

6. Now add recipients, subject, and content of the sent email along with the link to the shared Invoices folder on Drive

   

7. Select OK to confirm the changes in the module

8. Now set the schedule, for the Gmail module, click on the clock, and then schedule sending this email on the 10th day of the month

   

   

9. Save the module by clicking OK

10. Turn scheduling to ON and you’re done!

    

This is a super simple scenario, but introduces a nice concept of scheduling and running your robots remotely. If you want to expand it, try sending a Slack message to yourself with confirmation, after you send email. Small tip:

You already have your first automations on your account - congratulations! As you can see, it's child's play. Of course, these are just the basics, and in practice, automations can be much more complex, including, for example:

• Many more modules and routers, i.e., paths

• Conditional instructions "perform only when the data contains X"

• Functions, for example, searching through returned data

• Data completion, for example, with the current date or data calculation using mathematical operations

• Error handling and modules for testing

• Custom scripts, for example, JavaScript

• Modules allowing data reformatting (parsing)

• Modules capable of iterating, for example, over records in a database or rows in Google Sheets

• And so on

There are plenty of possibilities. But let's break down what really happens in the scenarios we created. The fact that Zapier or Make allows us to choose the Gmail module simply means that the tool can integrate with it. How? Of course, through APIs! For example, Gmail has an API, where we can refer to the appropriate endpoint and listen to incoming emails. Once we retrieve information about the incoming message, the next endpoint allows us to download the attachments. Zapier, which we use, just has these things built-in, but they work on the standard Gmail or Slack API. We work exactly like a programmer who would prepare an integration for us, except we use a visual builder and a set of prepared modules. The account connection we made is essentially the authentication in the API using one of the methods discussed earlier. It's programming without programming!

Make, like Zapier, offers the possibility to use hundreds of ready-made scenarios or import any blueprints prepared by us or others!

Automations & AI - the ultimate productivity tool

As you probably guess, over the years we've created quite a lot of automation for our needs. You might be surprised, but currently over 3000 robots work for me, performing hundreds of thousands of operations daily. This is just one of the Make accounts I use, where I process nearly half a million operations monthly!

But this is just the beginning! Just under a year ago, we realized how useful NLP and AI could be to help us work with automations. It turned out that input data for automations, which needed to be formatted in a specific way (we send them the so-called JSON object), could be prepared by AI! What does this mean in practice? It means that you can say or write, for example:

Add client John Doe to CRM

And then the language model will take care of translating this into the proper format, meaning into the "automation language," and then send it to a ready scenario on Zapier or Make! It's truly amazing how vast the possibilities are when using such an approach.

Unfortunately, there wasn't a tool that allowed us to communicate AI with automations in this way, so over a year ago, we started working on our own solution to help us with this.

And that's how Alice was born.

In addition to being able to chat with Alice like with ChatGPT, her biggest advantage lies in the ability to remotely trigger our automations, and consequently - perform actions! ChatGPT offers this concept also with GPTs and their API standard, however, this approach proved to be much less efficient so far, at least for our use-cases.

Future of remote actions in Alice is very promising, and probably we will, in time, start adding native integrations to Alice. Also, performance of LLMs is rapidly increasing, and also access to local models in Alice is a possibility. Today with Groq we’re able to perform operations faster than ever before, and this will evolve. Take a look at this command to Spotify and the speed of its execution.

In this context, I've prepared some tutorials for you, showing how you can integrate the Alice app with Make or Zapier. Please, see how it works in practice!

Remote Actions

Calendar scenario

Schedulink links scenario

Using existing scenarios

That’s it for today, but stay tuned! I’ve prepared for you a collection of inspirations and readymade automation blueprints that help us every day! Will be dropping it soon!

And one more thing. If you find today’s edition interesting, and you want to dig deeper into Alice, automations and our vision for productivity, we have something cooking for you! We’re considering creating a full-blown, 4-week AI+Automation cohort that you can take part in! There’s so much to learn, and here’s what we’re planning:

• 20 practical lessons — text + video (10+ hours of content)

• 4 live consultations (2h each)

• Very practical approach to AI (from the ground up)

• Showcase of the tools we already use on production

• Access to Alice and creating automations!

• Community-driven cohort and our support

If we manage to get 50 interested people (we have 7 so far!), we’ll start preparations!

Sure, count me in! »

See you in the next one!
Greg