◀ Course contents Part 2 · Module 2-02

GROUP BY

One number per category, instead of one for the table

The last module collapsed a whole table into a single number. This one does the same thing to each category separately, which is what turns an aggregate from a fact into a report. It also brings the one rule people break most often — and shows the error Postgres gives when they do.

Ready?

1

Sort Into Piles, Then Aggregate Each One

GROUP BY does exactly one thing: it sorts the rows into piles by the value of a column, and then runs every aggregate in your SELECT list once per pile instead of once for the whole table.

SELECT status, COUNT(*) AS n
FROM orders
GROUP BY status;

Three statuses in the table, three rows out. That is the rule worth internalising early: the number of rows you get back is the number of distinct values in the grouping column. You can predict it before you press Run, and when the answer surprises you, that surprise is usually the finding.

Rows sorted into piles, then aggregated per pile Twenty-four order rows are sorted by status into three piles — twenty-one paid, one pending, two refunded — and COUNT then runs once per pile, producing one summary row for each. orders 24 rows, each with a status paid, paid, refunded, paid, pending, paid… GROUP BY status pile: paid — 21 rows pile: pending — 1 row pile: refunded — 2 rows COUNT(*) per pile 3 rows out paid 21 pending 1 refunded 2 one row per pile
The piles are made first; the aggregate never sees the whole table. How many rows come back is settled before any counting happens — it is the number of different values in the column you grouped on.

ORDER BY still works and runs after the grouping, so it sorts the summary rows rather than the original ones. That is what lets you sort by an aggregate — and it is the only clause where an alias from the SELECT list is reliably visible, because by then the SELECT list has actually been computed.

SELECT device, COUNT(*) AS n
FROM events
GROUP BY device
ORDER BY n DESC;

Everyday example

A pile of receipts on a table. You could add them all up — that was the last module. Or you could first deal them into one pile per shop, then add each pile separately. The second way answers a different question, and the dealing has to happen before any adding does.

Quick check

A users table has 12 rows and 12 different countries. How many rows does SELECT country, COUNT(*) FROM users GROUP BY country return?

2

Grouped, Aggregated, or Not There at All

There is one rule, and it follows from what a group is. Each output row stands for a whole pile of input rows, so every column in the SELECT list must have exactly one value for that pile. Two ways to guarantee that:

1

A column you grouped by

Every row in the pile has the same value for it — that is what put them in the same pile.

2

A column inside an aggregate

SUM(amount), MAX(amount), COUNT(*). The aggregate is precisely the instruction for turning many values into one.

Anything else is a bare column, and it has no defensible answer. Ten orders in a pile, ten different amounts — which one should amount show?

SELECT user_id, amount, COUNT(*)
FROM orders
GROUP BY user_id;          -- amount: which of them?

Postgres refuses this outright, and names the column that caused it:

ERROR:  column "orders.amount" must appear in the
        GROUP BY clause or be used in an aggregate function

That error is doing you a favour, and it is worth going and triggering once in the editor so the wording is familiar. The query never runs, so it never misleads anyone.

MySQL and SQLite will answer it. They pick a value from some arbitrary row in the pile and hand it back without a word, which is much worse: the query runs, the number is plausible, nobody reviews it, and the value can change when the data or the query plan changes — which makes the eventual bug report read like witchcraft.

Rejected here

SELECT user_id, amount, COUNT(*)

A bare amount. Postgres names it and stops; the lenient databases answer with a row nobody chose. Same SQL, two very different outcomes.

Says what it means

SELECT user_id, MAX(amount), COUNT(*)

"The largest amount in this pile." One value, chosen on purpose, and correct on every database.

The query that changed its answer

A team prototyped a reporting query against a lenient local database and shipped it to a Postgres warehouse. It failed immediately with "column must appear in the GROUP BY clause" — which felt like the warehouse being difficult, and was in fact the first honest feedback the query had ever received. It had been returning an arbitrary row's value for months on the laptop it was written on.

Quick check

You group orders by user_id and also select ordered_at bare. What does Postgres do?

3

Several Columns, and Things You Compute

List two columns in GROUP BY and the group becomes the pair. Two rows share a pile only when both values match.

SELECT plan, is_active, COUNT(*) AS n
FROM users
GROUP BY plan, is_active;

Three plans and two activity states is six possible combinations — and four rows come back, because two of those combinations do not occur in the data. A group exists only where rows exist. There is no such thing as an empty group, which is the single most surprising thing about grouped results and the reason a report can quietly lose a category on a slow week.

You can also group by something computed rather than stored. The database works the expression out per row, then piles the rows up by its result:

SELECT EXTRACT(YEAR FROM ordered_at) AS yr, COUNT(*) AS n
FROM orders
GROUP BY EXTRACT(YEAR FROM ordered_at)
ORDER BY yr;

ordered_at is a real DATE, so a function pulls the year out of it — module 2-04 covers the rest of the date toolkit. The same trick with a CASE from module 1-06 gives you price tiers, age bands, or any bucket you care to invent; grouping is not limited to categories somebody already thought to store.

Repeat the expression, or group by position

The GROUP BY has to repeat the expression, because the alias is created in the SELECT list and that has not run yet — Postgres rejects GROUP BY yr with column "yr" does not exist. It does accept GROUP BY 1, meaning "the first output column", which is a genuine shorthand rather than a leniency. Repeating the expression is the version that reads best in a long query.

Quick check

Grouping sales by region and quarter: 4 regions, 4 quarters, but the Nordics only opened in Q3. How many rows?

4

WHERE Runs First, and Groups Can Vanish

The clause order is fixed and it is worth learning as a sentence: FROM, WHERE, GROUP BY, ORDER BY. Read a query in that order and grouped results stop being mysterious.

WHERE filters rows, on their way in, before any pile exists. A row it removes never reaches a group — and if every row of some group is removed, that group does not appear at all. It does not come back with a zero. It is simply not in the result.

SELECT user_id, COUNT(*) AS n
FROM orders
WHERE status = 'paid'
GROUP BY user_id;          -- 9 rows, not 10

Ten users have ordered. One of them has a single order and it was refunded, so WHERE removes their only row, their pile never forms, and they are missing from the report. That is usually correct — they have paid nothing — and occasionally it is the bug, when somebody downstream assumed every customer would have a line.

A fact about the row

WHERE status = 'paid'

Decidable by looking at one row on its own, so it belongs before the grouping.

A fact about the group

WHERE COUNT(*) > 3

Not answerable yet — the piles do not exist when WHERE runs. This needs a different clause, which is the next module.

That second case is the natural next question and it has its own answer: HAVING. Everything about it follows from the order of operations you have just learned, which is why it is next.

Quick check

A report groups orders by month. In August the shop was closed and there are no order rows at all. What does August look like in the result?

0 of 9 completed

Loading the tables…

01

To do

The last module collapsed a whole table to one number. GROUP BY does the same thing to each group separately: sort the rows into piles by some column, then run the aggregate once per pile.

SELECT status, COUNT(*) AS n
FROM orders
GROUP BY status;

Three statuses, three rows out. The number of rows in the result is the number of distinct values in the grouping column — which is a useful thing to predict before you run anything.

Your task: from products, return each category and how many products are in it, as category and n.

query.sql
PostgreSQL
Hint

SELECT category, COUNT(*) AS n FROM products GROUP BY category; — the grouping column goes in both the SELECT list and the GROUP BY.

Output

      
    02

    To do

    ORDER BY still works, and it runs after the grouping — so it sorts the summary rows, not the original ones. That means you can order by an aggregate, which is how every "top categories" list is built.

    You can sort by the alias you gave the aggregate. ORDER BY is one of the few places where an alias from the SELECT list is visible, because by then the SELECT list has been computed.

    Your task: from events, return each device and how many events came from it as n, most events first.

    query.sql
    PostgreSQL
    Hint

    SELECT device, COUNT(*) AS n FROM events GROUP BY device ORDER BY n DESC; — you can order by the alias n.

    Output
    
          
      03

      To do

      Any aggregate works per group, not just COUNT. Swap in SUM and each pile is added up instead of counted.

      Your task: from orders, return each user_id and the total amount they have spent as spent, biggest spender first.

      query.sql
      PostgreSQL
      Hint

      SELECT user_id, SUM(amount) AS spent FROM orders GROUP BY user_id ORDER BY spent DESC;

      Output
      
            
        04

        To do

        A group can carry as many aggregates as you like, and they all describe the same pile. This is how a one-line-per-category summary gets built, and it costs no more than a single one because the rows are only read once.

        Your task: from products, return each category, how many products it has as n, and the average price rounded to two decimals as avg_price — in that order, category name ascending.

        query.sql
        PostgreSQL
        Hint

        SELECT category, COUNT(*) AS n, ROUND(AVG(price), 2) AS avg_price FROM products GROUP BY category ORDER BY category;

        Output
        
              
          05

          To do

          List two columns and the group becomes the pair. Rows land in the same pile only when both values match, so the result has one row per combination that actually occurs.

          Combinations that never happen simply do not appear. There are three plans and two activity states here, which is six possible pairs — and four of them exist.

          Your task: from users, return plan, is_active and the count as n, ordered by plan then is_active.

          query.sql
          PostgreSQL
          Hint

          SELECT plan, is_active, COUNT(*) AS n FROM users GROUP BY plan, is_active ORDER BY plan, is_active; — a comma between the two grouping columns.

          Output
          
                
            06

            To do

            WHERE filters rows, before they are sorted into groups. So a row removed by WHERE never reaches its group, and a group whose rows are all removed disappears from the result entirely.

            That is the right tool when the condition is about a row. Filtering on the aggregate itself — "groups with more than three" — is a different clause, and it is the next module.

            Your task: from orders, counting paid orders only, return each user_id and the count as n, ordered by user_id.

            query.sql
            PostgreSQL
            Hint

            SELECT user_id, COUNT(*) AS n FROM orders WHERE status = 'paid' GROUP BY user_id ORDER BY user_id; — WHERE goes before GROUP BY, always.

            Output
            
                  
              07

              To do

              COUNT(DISTINCT ...) works per group too, and this is where it earns its keep: "how many different people did each thing" is a question about groups, and the row count answers it wrongly every time someone does the thing twice.

              Your task: from events, return each event_name, how many events there were as n, and how many different users did it as users — ordered by event_name.

              query.sql
              PostgreSQL
              Hint

              SELECT event_name, COUNT(*) AS n, COUNT(DISTINCT user_id) AS users FROM events GROUP BY event_name ORDER BY event_name;

              Output
              
                    
                08

                To do

                The rule for a grouped query is short: every column in the SELECT list must either be one you grouped by, or be wrapped in an aggregate. Anything else has one value per row and many rows per group, so there is no single answer for it.

                Postgres refuses such a query outright, and names the column:

                SELECT user_id, amount, COUNT(*)
                FROM orders
                GROUP BY user_id;

                ERROR:  column "orders.amount" must appear in the
                        GROUP BY clause or be used in an aggregate function

                That error is a favour. Ten orders in a pile, ten different amounts — there is no correct value for amount to show, so Postgres declines to invent one. MySQL and SQLite will answer this query, picking a value from an arbitrary row without comment, which is far worse: it runs, the number looks plausible, nobody reviews it, and it can change when the data does.

                Try it in the editor before you write the answer. Meeting the message once is what makes it readable the next time.

                Your task: the fix is to say what you meant. From orders, return each user_id, the count as n, and the largest amount that user has spent in one order as biggest — ordered by user_id.

                query.sql
                PostgreSQL
                Hint

                SELECT user_id, COUNT(*) AS n, MAX(amount) AS biggest FROM orders GROUP BY user_id ORDER BY user_id; — MAX is the aggregate that turns "which amount?" into a question with one answer.

                Output
                
                      
                  09

                  To do

                  You can group by an expression rather than a stored column. The database computes it per row, then piles the rows up by the result — which is how you group by month, by first letter, or by a bucket you invent with the CASE from module 1-06.

                  ordered_at is a real DATE, so the year comes out of it with a function rather than by slicing characters: EXTRACT(YEAR FROM ordered_at) gives the number 2024. Module 2-04 covers the rest of the date toolkit.

                  Your task: from orders, return the year as yr and the number of orders in it as n, oldest year first.

                  query.sql
                  PostgreSQL
                  Hint

                  SELECT EXTRACT(YEAR FROM ordered_at) AS yr, COUNT(*) AS n FROM orders GROUP BY EXTRACT(YEAR FROM ordered_at) ORDER BY yr; — repeat the expression in the GROUP BY, or write GROUP BY 1.

                  Output
                  
                        

                    Revenue by status

                    To do

                    Finance wants one line per order status: how many, how much, and what the typical order looked like. They want to see every status, including the ones that are not revenue — the whole point is to show what the headline number is leaving out.

                    Your task: from orders, grouped by status, return exactly these four columns in this order:

                    • status — the group
                    • n — how many orders have it
                    • revenue — the total amount
                    • avg_order — the average amount, rounded to two decimals

                    Sorted by revenue, largest first. No WHERE clause — every status appears.

                    query.sql
                    PostgreSQL
                    Hint

                    SELECT status, COUNT(*) AS n, SUM(amount) AS revenue, ROUND(AVG(amount), 2) AS avg_order FROM orders GROUP BY status ORDER BY revenue DESC;

                    Output
                    
                          

                      Notification