gmailctl

This utility helps you generate and maintain Gmail filters in a declarative way. It has a Jsonnet configuration file that aims to be simpler to write and maintain than using the Gmail web interface, to categorize, label, archive and manage your inbox automatically.

Table of contents

• gmailctl
  • Table of contents
  • Motivation
  • Install
  • Usage
    • Config directory
    • Migrate from another solution
    • Other commands
  • Configuration
    • Search operators
    • Logic operators
    • Reusing filters
    • Actions
    • Labels
    • Tests
  • Tips and tricks
    • Chain filtering
    • To me
    • Directly to me
    • Automatic labels
    • Multiple Gmail accounts
  • Known issues
    • Apply filters to existing emails
    • OAuth2 authentication errors
    • YAML config is unsupported
  • Comparison with existing projects
  • Footnotes

Motivation

If you use Gmail and have to maintain (like me) a lot of filters (to apply labels, get rid of spam or categorize your emails), then you probably have (like me) a very long list of messy filters. At a certain point one of your messages got mislabled and you try to understand why. You scroll through that horrible mess of filters, you wish you could find-and-replace stuff, test the changes on your filters before applying them, refactor some filters together... in a way treat them like you treat your code!

Gmail allows one to import and export filters in XML format. This can be used to maintain them in some better way... but dear Lord, no! Not by hand! That's what most other tools do: providing some kind of DSL that generate XML filters that can be imported in your settings... by hand [this is the approach of the popular antifuchs/gmail-britta for example].

Gmail happens to have also a neat API that we can use to automate the import step as well, so to eliminate all manual, slow tasks to be done with the Gmail settings.

This project then exists to provide to your Gmail filters:

1. Maintainability;
2. An easy to understand, declarative, composable language;
3. A builtin query simplifier, to keep the size of your filters down (Gmail has a limit of 1500 chars per filter);
4. Ability to review your changes before applying them;
5. Automatic update of the settings (no manual import) in seconds.

Install

Pre-built binaries for Linux, macOS, and Windows (amd64 and arm64) are available on the GitHub releases page. Download the archive for your platform, extract it, and place the gmailctl binary somewhere in your $PATH.

Alternatively, if you have Go installed, you can build and install from source:

gmailctl is written in Go and requires a recent version (see go.mod). Make sure to setup your $GOPATH correctly and include its bin subdirectory in your $PATH.

go install github.com/mbrt/gmailctl/cmd/gmailctl@latest

If you're on macOS, you can also install via Homebrew or Macports:

# Install with Homebrew
brew install gmailctl

# Install with Macports
sudo port install gmailctl

On Fedora Linux, you can install from the official repositories:

sudo dnf install gmailctl

You can also choose to install the snap:

sudo snap install gmailctl

If so, make sure to configure xdg-mime to open the config file with your favorite editor. For example, if you'd like to use vim:

xdg-mime default vim.desktop text/x-csrc

If you are on windows, you can configure an editor (like VS Code) like this:

//cmd
set "EDITOR=code --wait"

//powershell
setx EDITOR "code --wait"

Once installed, run the init process:

gmailctl init

This will guide you through setting up the Gmail APIs and update your settings without leaving your command line.

Usage

The easiest way to use gmailctl is to run gmailctl edit. This will open the local config file in your editor. After you exit the editor the configuration is applied to Gmail. See Configuration for the configuration file format. This is the preferred way if you want to start your filters from scratch.

NOTE: It's recommended to backup your current configuration before you apply the generated one for the first time. Your current filters will be wiped and replaced with the ones specified in the config file. The diff you'll get during the first run will probably be pretty big, but from that point on, all changes should generate a small and simple to review diff.

Config directory

Configuration and credentials are in either:

• <XDG_BASE_DIR>/gmailctl (using the XDG base directory spec, commonly ~/.config/gmailctl on Linux.
• ~/.gmailctl: The previous default location. If the directory exists gmailctl will continue to use it for backward compatibility
• Custom location: use the --config argument.

  Migrate from another solution

If you want to preserve your current filters and migrate to a more sane configuration gradually, you can try to use the download command. This will look up at your currently configured filters in Gmail and try to create a configuration file matching the current state.

NOTE: This functionality is experimental. It's recommended to download the filters and check that they correspond to the remote ones before making any changes, to avoid surprises. Also note that the configuration file will be quite ugly, as expressions won't be reconstructed properly, but it should serve as a starting point if you are migrating from other systems.

Example of usage:

# download the filters to the default configuration file
mkdir -p ~/.config/gmailctl
gmailctl download > ~/.config/gmailctl/config.jsonnet
# check that the diff is empty and no errors are present
gmailctl diff
# happy editing!
gmailctl edit

Often you'll see imported filters with the isEscaped: true marker. This tells gmailctl to not escape or quote the expression, as it might contain operators that have to be interpreted as-is by Gmail. This happens when the download command was unable to map the filter to native gmailctl expressions. It's recommended to manually port the filter to regular gmailctl operators before doing any changes, to avoid unexpected results. Example of such conversion:

{
  from: "{foo bar baz}",
  isEscaped: true,
}

Can be translated into:

{
  or: [
    {from: "foo"},
    {from: "bar"},
    {from: "baz"},
  ],
}

Other commands

All the available commands (you can also check with gmailctl help):

  apply       Apply a configuration file to Gmail settings
  debug       Shows an annotated version of the configuration
  diff        Shows a diff between the local configuration and Gmail settings
  download    Download filters from Gmail to a local config file
  edit        Edit the configuration and apply it to Gmail
  export      Export filters into the Gmail XML format
  help        Help about any command
  init        Initialize the Gmail configuration
  test        Execute config tests

Configuration

NOTE: Despite the name, the configuration format is stable at v1alpha3. If you are looking for the deprecated versions v1alpha1, or v1alpha2, please refer to docs/v1alpha1.md and docs/v1alpha2.md.

The configuration file is written in Jsonnet, that is a very powerful configuration language, derived from JSON. It adds functionality such as comments, variables, references, arithmetic and logic operations, functions, conditionals, importing other files, parameterizations and so on. For more details on the language, please refer to the official tutorial.

Simple example:

// Local variables help reuse config fragments
local me = {
  or: [
    { to: 'pippo@gmail.com' },
    { to: 'pippo@hotmail.com' },
  ],
};

// The exported configuration starts here
{
  version: 'v1alpha3',
  // Optional author information (used in exports).
  author: {
    name: 'Pippo Pluto',
    email: 'pippo@gmail.com'
  },
  rules: [
    {
      filter: {
        and: [
          { list: 'geeks@newsletter.com' },
          { not: me },  // Reference to the local variable 'me'
        ],
      },
      actions: {
        archive: true,
        labels: ['news'],
      },
    },
  ],
}

The Jsonnet configuration file contains mandatory version information, optional author metadata and a list of rules. Rules specify a filter expression and a set of actions that will be applied if the filter matches.

Filter operators are prefix of the operands they apply to. In the example above, the filter applies to emails that come from the mail list 'geeks@newsletter.com' AND the recipient is not 'me' (which can be 'pippo@gmail.com' OR 'pippo@hotmail.com').

We will see all the features of the configuration file in the following sections.

Search operators

Search operators are the same as the ones you find in the Gmail filter interface:

• from: the mail comes from the given address
• to: the mail is delivered to the given address
• subject: the subject contains the given words
• has: the mail contains the given words

In addition to those visible in the Gmail interface, you can specify natively the following common operators:

• list: the mail is directed to the given mail list
• cc: the mail has the given address as CC destination
• bcc: the mail has the given address as BCC destination
• replyto: the mail has the given address as Reply-To destination

One more special function is given if you need to use less common operators<sup id="a1">1, or want to compose your query manually:

• query: passes the given contents verbatim to the Gmail filter, without escaping or interpreting the contents in any way.

Example:

{
  version: 'v1alpha3',
  rules: [
    {
      filter: { subject: 'important mail' },
      actions: {
        markImportant: true,
      },
    },
    {
      filter: {
        query: 'dinner AROUND 5 friday has:spreadsheet',
      },
      actions: {
        delete: true,
      },
    },
  ],
}

Logic operators

Filters can contain only one expression. If you want to combine multiple of them in the same rule, you have to use logic operators (and, or, not). These operators do what you expect:

• and: is true only if all the sub-expressions are also true
• or: is true if one or more sub-expressions are true
• not: is true if the sub-expression is false.

Example:

{
  version: 'v1alpha3',
  rules: [
    {
      filter: {
        or: [
          { from: 'foo' },
          {
            and: [
              { list: 'bar' },
              { not: { to: 'baz' } },
            ],
          },
        ],
      },
      actions: {
        markImportant: true,
      },
    },
  ],
}

This composite filter marks the incoming mail as important if:

• the message comes from "foo", or
• it is coming from the mailing list "bar" and not directed to "baz"

Reusing filters

Filters can be named and referenced in other filters. This allows reusing concepts and so avoid repetition. Note that this is not a gmailctl functionality but comes directly from the fact that we rely on Jsonnet.

Example:

local toMe = {
  or: [
    { to: 'myself@gmail.com' },
    { to: 'myself@yahoo.com' },
  ],
};
local notToMe = { not: toMe };

{
  version: 'v1alpha3',
  rules: [
    {
      filter: {
        and: [
          { from: 'foobar' },
          notToMe,
        ],
      },
      actions: {
        delete: true,
      },
    },
    {
      filter: toMe,
      actions: {
        labels: ['directed'],
      },
    },
  ],
}

In this example, two named filters are defined. The toMe filter gives a name to emails directed to 'myself@gmail.com' or to 'myself@yahoo.com'. The notToMe filter negates the toMe filter, with a not operator. Similarly, the two rules reference the two named filters above. The name reference is basically copying the definition of the filter in place.

The example is effectively equivalent to this one:

{
  version: 'v1alpha3',
  rules: [
    {
      filter: {
        and: [
          { from: 'foobar' },
          {
            not: {
              or: [
                { to: 'myself@gmail.com' },
                { to: 'myself@yahoo.com' },
              ],
            },
          },
        ],
      },
      actions: {
        delete: true,
      },
    },
    {
      filter: {
        or: [
          { to: 'myself@gmail.com' },
          { to: 'myself@yahoo.com' },
        ],
      },
      actions: {
        labels: ['directed'],
      },
    },
  ],
}

Relying on Jsonnet also allows importing code and raw data from other files³.

Actions

Every rule is a composition of a filter and a set of actions. Those actions will be applied to all the incoming emails that pass the rule's filter. These actions are the same as the ones in the Gmail interface:

• archive: true: the message will skip the inbox;
• delete: true: the message will go directly to the trash can;
• markRead: true: the message will be mark as read automatically;
• star: true: star the message;
• markSpam: false: do never mark these messages as spam. Note that setting this field to true is not supported by Gmail (I don't know why);
• markImportant: true: always mark the message as important, overriding Gmail heuristics;
• markImportant: false: do never mark the message as important, overriding Gmail heuristics;
• category: <CATEGORY>: force the message into a specific category (supported categories are "personal", "social", "updates", "forums", "promotions", "purchases");
• labels: [list, of, labels]: an array of labels to apply to the message. Note that these labels have to