Firewalls

Most production machines run a firewall daemon by default. Ask an engineer what it is doing, and the standard reply is usually “blocking bad stuff”—which is true in the aggregate, but useless when trying to determine why a single packet on port 8080 was dropped. Given sufficient request volume and multi-hop overlay networks, the gap between “blocking bad stuff” and predicting a packet’s fate across an ordered chain manifests as production incidents: healthy VPN tunnels that carry zero traffic, or remote SSH sessions that instantly terminate the second ufw enable executes.

I’m going to make the case that a firewall is best understood through what I call The Guest-List Model: an ordered list of questions asked about each packet from top to bottom, paired with a fallback policy for unlisted traffic.

Two critical behaviors follow from this model:

1. The default policy is the actual security boundary; every explicit rule is merely an exception to it.
2. The first rule that matches a packet wins—the exact opposite of IP routing, which selects the narrowest subnet match.

What a firewall is, in networking terms

Before we configure anything, the vocabulary. Each term gets used a lot, so each gets a definition now.

• packet — a unit of network traffic, carrying a source address, a destination address, and a little bookkeeping about which conversation it belongs to.
• port — a number that says which application on a machine should get this packet, like an apartment number on a street address. Port 22 is SSH, port 80 is HTTP, port 443 is HTTPS.
• inbound — traffic arriving at the machine from somewhere else.
• outbound — traffic leaving the machine toward somewhere else.
• rule — one entry in the list: a condition, like “destination port 80,” and a verdict, like “allow.”
• policy — the default verdict, the answer given when no rule matched.
• frontend — a simpler tool that writes the complicated rules for you.

That last one matters because it frames the whole practical half of the article. The thing doing the actual work in Linux is not a program you talk to directly. It is the kernel’s packet-filtering machinery, called netfilter, which reads a ruleset that most humans find unpleasant to write by hand. For decades the way you wrote that ruleset was a tool called iptables, and more recently a cleaner one called nftables. Both are powerful, and both are easy to get wrong in ways that fail silently.

Which brings us to the third kind of tool, and the one this article is really about.

The three kinds of firewall

You will hear “firewall” used for at least three different things, and conflating them causes real confusion, so let’s separate them once, cleanly.

A traditional firewall, also called a packet filter, looks at each packet’s header — source, destination, port, protocol — and decides allow or deny based on rules. In its modern form it is stateful, meaning it remembers the conversations it has already approved, so it can let the replies back in without a separate rule for them. This is what netfilter is, what iptables and nftables drive, and what UFW sits on top of.

A proxy firewall does not forward your packets at all. It terminates your connection, inspects the request at the application layer, and opens its own separate connection to the destination on your behalf. Nothing on either side ever talks to the other directly. The trade is latency and complexity for a much deeper look at the content.

A next-generation firewall, or NGFW, is the traditional packet filter plus a bundle of additions: application awareness that knows it is looking at a specific app and not just port 443, user identity, and usually intrusion prevention, which is the firewall watching for known attack patterns rather than just checking addresses. These are what enterprises buy as appliances.

The packet’s journey

The guest-list model is easier to watch than to describe, so the figure for this section is animated. It shows a vertical stack of rules read top to bottom, and two packets arriving.

One packet is addressed for port 80. It does not match the first rule, matches the second, and is allowed through. The other is addressed for port 25. It matches nothing, falls through to the default, and is denied.

Caption for the figure:

A firewall reads its rules top to bottom and stops at the first match. Only if nothing matches does the default policy decide.

The SVG source for this figure is at the end of the article.

The Default policy

The default policy is the answer for everything nobody thought about, which is why it matters more than any individual rule. In practice, people choose one of three, and the choice says a lot about what the machine is for.

Deny inbound, allow outbound. The machine can start conversations and receive their replies, but nobody outside can start one with it. This is the common, sensible starting point for servers and laptops alike, and it is what UFW ships with. It is also what this article uses.

Allow everything. The convenience default of most home routers. Fine behind a trusted network, and a problem the moment the machine meets an untrusted one.

Deny everything. The locked-down posture, where even outbound must be invited. You see it in environments where the goal is not convenience but auditability.

The recommendation is to start closed to inbound, open to outbound, and treat every allow rule as a deliberate exception you can name and justify.

On Ubuntu and Debian, the standard frontend is UFW, which stands for Uncomplicated Firewall, and for once the name is accurate. You tell it your intent in sentences, and it writes the netfilter rules underneath.

sudo ufw default deny incoming
sudo ufw allow 80/tcp
sudo ufw enable

The important thing to understand about UFW is that it is a thin, opinionated layer. It is not a different firewall. Underneath it is still netfilter, still first-match, still a default policy. Everything in the first half of this article applies unchanged. UFW just means you express it in commands you can read back.

A firewall lives in the kernel, and a container shares the host’s kernel, so running UFW inside a container works, but it is the part of this lab most likely to behave differently across podman and docker versions, and across rootless versus rootful setups. If a command below fights you, that is the reason, and the fix is usually to run the container rootful. The concepts don’t change.

Building the two-node lab

Let’s build a small, disposable laboratory to demonstrate this. Rather than reading firewall rules on a host machine where a misstep can sever our own access, we will instantiate two lightweight containers on an isolated bridge network.

$ podman network create labnet

# terminal one: the machine that will run the firewall
$ podman run --rm -it --name fw-server --network labnet \
    --cap-add NET_ADMIN docker.io/library/ubuntu:24.04 bash

# terminal two: the machine that will probe it
$ podman run --rm -it --name fw-client --network labnet \
    docker.io/library/ubuntu:24.04 bash

We grant --cap-add NET_ADMIN solely to fw-server so it can manipulate kernel packet-filtering tables via UFW. The client is just a burglar with curl. Inside fw-server, we install UFW and Python:

Now give each machine its tools. On the server:

root@fw-server:# apt update && apt install -y ufw python3

On the client:

root@fw-client:# apt update && apt install -y curl netcat-openbsd

And find out where they live. On each:

root@fw-server:# hostname -i
10.89.0.2
root@fw-client:# hostname -i
10.89.0.3

I’ll refer to the server as 10.89.0.2 and the client as 10.89.0.3. Substitute whatever your containers actually report.

Baseline Connection vs. Inactive State

Before applying constraints, we observe the unencumbered path. On fw-server, we start an HTTP daemon on port 80:

root@fw-server:# python3 -m http.server 80 &

From fw-client, we execute a GET request:

root@fw-client:# curl -m 3 -o /dev/null -w '%{http_code}\n' http://10.89.0.2/
200

The 200 response serves as our baseline. Querying UFW status on fw-server confirms why:

root@fw-server:# ufw status
Status: inactive

When netfilter has no active UFW rules loaded, the Linux kernel defaults to forwarding and accepting inbound packets.

Setting Policy Before Rule Activation

A frequent operational mistake is enabling a firewall before defining its default fallback policy. We establish our target posture while the engine remains inactive:

root@fw-server:# ufw default deny incoming
Default incoming policy changed to 'deny'

root@fw-server:# ufw default allow outgoing
Default outgoing policy changed to 'allow'

Diagnose: Evaluating the draft configuration

root@fw-server:# ufw status verbose
Status: inactive
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

The default policy dictates what occurs when a packet reaches the bottom of our list without triggering a match. I call this posture Default-Deny Inbound. By allowing outbound while dropping inbound, the machine retains the ability to initiate socket connections (such as fetching OS packages) while rejecting unsolicited external connections.

Inviting Traffic and the Lockout Mechanics

Here is where the container lab differs from a real server, and the difference is instructive. On a real server you would allow SSH now, before enabling, because SSH is your way back in and the firewall can take it away. In this lab our way in is the attached terminal, which is not network traffic, so the firewall cannot touch it. The classic lockout cannot happen here.

That is worth sitting with for a moment, because it is the whole lockout lesson in a safer form. The reason “allow your access method before you enable” is a rule is that the access method is itself inbound traffic. Our terminal is not, so we are immune. A remote SSH session is, so it is not. Same rule, and now you know why it is a rule instead of just memorizing it.[^1]

So we allow the web service, then enable.

root@fw-server:# ufw allow 80/tcp
Rules updated
Rules updated (v6)

root@fw-server:# ufw enable
Firewall is active and enabled on system startup

Diagnose:

root@fw-server:# ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 80/tcp                     ALLOW IN    Anywhere
[ 2] 80/tcp (v6)                ALLOW IN    Anywhere (v6)

Note the IPv6 twin. UFW usually adds one. I’ll omit the v6 lines in later output for readability, but yours will show them.

Now the real test, from the client:

root@fw-client:# curl -m 3 -o /dev/null -w '%{http_code}\n' http://10.89.0.2/
200

The service is reachable through an active firewall, because we invited it. This is the moment to notice what we did not do: we never opened anything else. Every other inbound port is still denied by default.

To see that concretely, probe a port we did not invite. Port 81 has nothing on it, and no rule for it.

root@fw-client:# curl -m 3 -o /dev/null -w '%{http_code}\n' http://10.89.0.2:81/
curl: (28) Operation timed out after 3001 milliseconds with 0 bytes received

A timeout, not a refusal. Port 81 has no explicit rule, that is the default policy answering for something nobody put on the list. Keep that timeout in mind; we’re going to make it say something different in a moment.

The Guest-List Inversion (First-Match vs. Most-Specific)

Network Engineers accustomed to IP routing often assume firewalls prioritize narrow subnet definitions over broader ones (Longest Prefix Match). Firewall engines do not behave this way. They evaluate rules sequentially from top to bottom and terminate evaluation immediately upon the first match.

To demonstrate this edge case, we insert an explicit deny rule for 10.89.0.3 at position 1, above the global allow rule for port 80:

root@fw-server:# ufw insert 1 deny from 10.89.0.3 to any port 80
Rule inserted
Rule inserted (v6)

Diagnose: Inspecting the numbered table

root@fw-server:# ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 80                         DENY IN     10.89.0.3
[ 2] 80/tcp                     ALLOW IN    Anywhere
[ 3] 80/tcp (v6)                ALLOW IN    Anywhere (v6)

There are now two rules about port 80. One says deny our client, one says allow anyone. A routing brain expects the more specific one to win. A firewall does not do that. It reads top to bottom, hits rule one, and stops.

From fw-client, we reissue the curl command:

root@fw-client:# curl -m 3 -o /dev/null -w '%{http_code}\n' http://10.89.0.2/
curl: (28) Operation timed out after 3001 milliseconds with 0 bytes received

Denied, even though an allow rule for port 80 exists, because the deny came first. This is the single most common firewall misunderstanding, and you just watched it happen.

Clean it up by removing rule one.

root@fw-server:# ufw delete 1
Deleting:
 deny from 10.89.0.3 to any port 80
Proceed with operation (y|n)? y
Rule deleted

And confirm the service is reachable again:

root@fw-client:# curl -m 3 -o /dev/null -w '%{http_code}\n' http://10.89.0.2/
200

Drop Semantics vs. Reject Semantics

When denying traffic, firewalls offer two distinct verdicts: DROP and REJECT.

• DROP (Silent Dropping): Discards the packet without returning a response. The client cannot distinguish between a firewall block and a dead host or blackhole route, causing sockets to hang until timeouts expire.
• REJECT (Immediate Rejection): Discards the packet and emits an explicit ICMP Destination Unreachable or TCP RST frame back to the client.

We configure port 2525 with an explicit reject action:

root@fw-server:# ufw reject 2525/tcp
Rules updated
Rules updated (v6)

We now compare the terminal output between a rejected port (2525) and a dropped port (26, handled by default policy):

root@fw-client:# nc -w 3 -vz 10.89.0.2 2525
nc: connect to 10.89.0.2 port 2525 (tcp) failed: Connection refused

root@fw-client:# nc -w 3 -vz 10.89.0.2 26
nc: connect to 10.89.0.2 port 26 (tcp) failed: Connection timed out

Read those two lines carefully. Same firewall, two different refusals. The reject said “no” fast. The drop said nothing and let the client burn three seconds finding out.

Which do you want? It depends on who is asking. For random internet noise, a drop is usually better; it gives away nothing and wastes the scanner’s time. For your own clients, a reject is kinder, because a fast “refused” is easier to debug than a silent timeout. There is no universal answer, only a deliberate choice, and now you know what you are choosing between.

The difference is observable in client telemetry: Connection refused indicates an explicit rejection, while Connection timed out indicates a dropped frame. As a design rule, drop Internet-facing noise to starve port scanners of state data, but use reject on internal network segments so microservices fail fast rather than hanging indefinitely.

Connection tracking: how replies get back in

There is a question the defaults raised that we have not answered. We set inbound to deny. Then we ran curl from the server earlier and, in the next section’s terms, we will see outbound traffic complete. If inbound is denied, how does the reply to our own outbound request get back in?

The answer is the piece that makes a modern firewall stateful. Let’s prove it with the lab.

On the client, start a service.

root@fw-client:# python3 -m http.server 8080 &

Now, from the server, fetch it. Remember, the server’s inbound policy is deny.

root@fw-server:# curl -m 3 -o /dev/null -w '%{http_code}\n' http://10.89.0.3:8080/
200

The request succeeds with a 200 status. This occurs because modern packet filters utilize the Linux kernel’s conntrack (connection tracking) module. Netfilter automatically tracks outbound TCP handshakes and marks returning inbound frames with the ESTABLISHED state.

We can inspect the underlying rule UFW injects into netfilter to verify this mechanism directly:

root@fw-server:# grep -n 'ESTABLISHED' /etc/ufw/user.rules | head -n1
-A ufw-before-input -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

That line, which UFW wrote and which we never touched, is the whole idea: packets belonging to ESTABLISHED or RELATED conversations are accepted before the guest list is even consulted. This is why you do not need an inbound allow rule for every reply, and it is why “stateful” is not a buzzword but the specific mechanism that makes default-deny-inbound livable.

Because stateful rules evaluate in UFW’s early-input chain prior to user-defined guest lists, returning response packets bypass default-deny evaluation entirely.

Operational Hygiene

Configuring rulesets is straightforward; maintaining guest list integrity across large deployments requires strict operational patterns:

1. Maintain Default-Deny Posture: Never change default incoming policies to allow for debugging convenience.
2. Audit Rule Positionally: Periodically run ufw status numbered to prune stale exceptions and verify top-to-bottom precedence.
3. Scope Source Subnets: Avoid broad port allowances (allow 80/tcp). Prefer source-restricted definitions (allow from 10.89.0.3 to any port 80 proto tcp).
4. Document Rule Provenance: Firewalls store network criteria, not business intent. Maintain a revision-controlled log mapping rule indices to ticket IDs and review dates.

Tear Down

Because we constructed our test environment within ephemeral container boundaries, cleanup requires no rule resetting or state restoration on the host:

root@fw-server:# exit
root@fw-client:# exit
$ podman network rm labnet

Exiting the containers removes the filesystems and kernel networking namespaces completely.