Firewalls and IDS/IPS

How firewalls (packet filter, stateful, proxy, NGFW, WAF) and intrusion detection/prevention systems work, where to place them, how to write and tune rules, and what they cannot see.

A firewall decides which traffic is allowed to flow between two networks or into a host. An IDS/IPS watches traffic (or host activity) for signs of attack. The firewall is the gatekeeper that enforces policy; the IDS/IPS is the inspector that looks for abuse inside traffic the policy allowed. Neither is enough alone, and both belong to the layered approach described in Security Architecture Principles.


Firewalls

Generations

TypeWorks atDecides usingStrengthWeakness
Packet filter (stateless)L3/L4Source/destination IP, port, protocol, flags, per packetFast, simpleNo memory of connections; easy to spoof or abuse return traffic
Stateful inspectionL3/L4Same fields plus a connection state table (NEW, ESTABLISHED, RELATED)Allows replies automatically; the default for most firewallsDoes not understand the application payload
Proxy / application gatewayL7Terminates the client connection and opens a new one to the serverDeep protocol checks, hides internal hosts, good loggingSlower, one proxy per protocol
NGFWL3-L7Stateful plus application identification, user identity, IPS, TLS inspection, threat feedsPolicy by app and user, not just portCost, complexity, performance impact when all features are on
WAFL7 (HTTP/S)HTTP requests and responses in front of a web appBlocks injection, XSS and bots, virtual patchingNot a general firewall; bypassable, needs tuning

A WAF protects one web application and complements, rather than replaces, secure code (see Application Attacks).

Rule anatomy and ordering

A rule is a match condition plus an action. Typical fields: source, destination, protocol, source/destination port, interface or direction, action (allow, deny/drop, reject, forward), and logging.

text
# order  action  proto  source          dest            dport   log
  10     ALLOW   tcp    10.10.20.0/24   DMZ-web         443     yes
  20     ALLOW   tcp    DMZ-web         10.10.30.5      5432    yes
  30     DENY    any    any             any             any     yes   <- default deny

  • Rules are evaluated top to bottom, first match wins. Specific rules go above general ones, and a broad allow placed too high silently overrides everything beneath it.
  • Default deny: end with an explicit deny-all and allow only what is needed. The opposite (default allow) means every mistake is an open door.
  • DROP vs REJECT: drop silently discards (slows scanners, confuses legitimate clients); reject sends an error back (faster failure, reveals the host exists).
  • Log denies, and log allows for sensitive paths. Review rules regularly; unused and overly broad rules accumulate over time (see Change Management).

Ingress and egress filtering

  • Ingress: controls traffic coming in. Block spoofed internal source addresses, private/bogon ranges arriving from the internet, and all services that should not be public.
  • Egress: controls traffic leaving. Often neglected, yet it limits what malware can do after a compromise: command-and-control callbacks, data exfiltration and reverse shells all need to leave. Allow only required outbound ports and force web traffic through a proxy or DNS filter (see Security Technologies).

Host vs network firewalls

Network firewallHost firewall
LocationAppliance, VM or cloud security group between segmentsSoftware on each endpoint (Windows Defender Firewall, iptables/nftables, ufw, macOS pf)
ProtectsWhole segments, north-south trafficOne machine, including east-west traffic that never crosses the perimeter
LimitCannot see traffic inside a segmentNeeds central management to stay consistent

Use both. A host firewall is what stops lateral movement inside a flat network (see Network Segmentation).

Placement: DMZ / screened subnet

text
                      Internet
                          |
                 +--------+--------+
                 |  Edge firewall  |
                 +---+---------+---+
                     |         |
             DMZ (screened)    |
          +----------+------+  |
          | web / mail / DNS|  |
          | reverse proxy   |  |
          +----------+------+  |
                     |         |
                 +---+---------+---+
                 | Internal firewall|
                 +--------+--------+
                          |
                  Internal networks
                 (users, servers, DB)

Internet-facing services live in the DMZ. The internet may reach the DMZ, the DMZ may reach only specific internal services, and the internal network is never reachable directly from outside. Two firewalls (ideally different vendors), or one with three interfaces, both work.

NAT note

NAT (and PAT) rewrites addresses so many private hosts share a public IP. It hides internal addressing and drops unsolicited inbound connections as a side effect, but it is not a security control by itself. It is address translation, not policy. Rely on firewall rules, not on "it is behind NAT".

Example tools

  • iptables / nftables: Linux kernel packet filtering (nftables is the modern replacement).
  • pfSense / OPNsense: open-source firewall distributions with stateful filtering, VPN and optional IDS packages.
  • Windows Defender Firewall: built-in host firewall with network profiles.
  • Cloud: security groups and network ACLs (see Cloud Security).

bash
# nftables concept: default-drop inbound, allow established and SSH from management net
table inet filter {
  chain input {
    type filter hook input priority 0; policy drop;
    ct state established,related accept
    iif lo accept
    ip saddr 10.10.99.0/24 tcp dport 22 accept
  }
}


IDS and IPS

  • IDS (Intrusion Detection System): passive, observes a copy of traffic or logs and alerts.
  • IPS (Intrusion Prevention System): sits inline, and can block or drop malicious traffic in real time.

An IPS can stop an attack but can also break legitimate traffic and becomes a single point of failure. Many deployments start in detect-only mode and move rules to prevent once they are trusted.

NIDS vs HIDS

NIDS (network)HIDS (host)
WatchesPackets crossing a network pointLogs, file integrity, processes, system calls on one host
SeesMany hosts at once, including unmanaged devicesLocal activity, even inside encrypted sessions
Blind toEncrypted payloads, activity on the host itselfOther hosts, network-wide patterns
ExamplesSnort, Suricata, Zeek (as a network analysis framework)OSSEC/Wazuh, file integrity monitoring, EDR (see Endpoint Security)

Detection methods

MethodHowProsCons
Signature-basedMatch known patterns/rulesAccurate for known threats, easy to explainMisses new or modified attacks
Anomaly-basedCompare to a learned baseline of normalCan catch unknown attacksNoisy; needs a good baseline
HybridBothBalancedMore to maintain

Placement: inline, SPAN, tap

text
Inline (IPS)                     Passive (IDS)

 Internet                         Internet
    |                                |
[Firewall]                      [Firewall]
    |                                |
 [ IPS ]  <- traffic passes       [Switch] --SPAN/mirror--> [IDS sensor]
    |        through it             |
 Internal                        Internal

  • Inline: can block, but adds latency and a failure point (use fail-open/bypass carefully).
  • SPAN/mirror port: the switch copies traffic to a sensor; cheap, but can drop packets under load.
  • Network TAP: a hardware device that copies traffic faithfully without burdening the switch.
  • Place sensors at the perimeter, in front of critical zones, and between internal segments, not just at the edge.

False positives and negatives

ResultMeaningImpact
False positiveAlert on benign trafficAlert fatigue, wasted analyst time, blocked users (IPS)
False negativeReal attack not detectedUndetected compromise, the dangerous one
True positive / negativeCorrect detection / correct silence

Reducing one tends to raise the other, so tune deliberately: disable rules irrelevant to your environment, adjust thresholds, suppress known-good sources, and test changes against captured traffic before enabling blocking. Feed alerts into a SIEM for correlation (SIEM and Log Management, Security Operations Center).

Example rule (Snort/Suricata style)

text
alert tcp any any -> $HOME_NET 22 (msg:"Possible SSH brute force";
    flags:S; threshold:type both, track by_src, count 5, seconds 60;
    sid:1000001; rev:1;)

Structure: action, protocol, source, direction, destination, then options in parentheses. The TryHackMe note covers writing and testing Snort rules in detail.


Evasion awareness and blind spots

Attackers know these controls exist. Defenders should know where they fail.

  • Fragmentation and obfuscation: splitting payloads across fragments or segments, or encoding them, can hide a signature unless the device reassembles and normalises traffic first. Scanners use similar tricks (see Nmap and Network Attacks).
  • Encryption: HTTPS, SSH and VPN tunnels hide payloads from NIDS and basic firewalls (VPN and Tunneling, SSH Advanced). You see metadata only (IPs, SNI, sizes, timing).
  • Allowed ports as cover: malware tunnelling over 443 or DNS passes a port-based firewall; this is why NGFW app-ID, egress control and DNS filtering matter.
  • Slow and low: scans or brute force spread over hours stay under thresholds.

TLS inspection trade-offs

Decrypting TLS at a proxy/NGFW restores visibility, but:

BenefitCost
Detect malware and exfiltration inside HTTPSCPU load and latency
Enforce DLP and content policyThe inspection device holds a trusted CA and sees all plaintext, so it is a high-value target
Privacy and legal concerns; exempt banking/health categories
Breaks certificate pinning and some applications

Key points

  • Stateful firewalls are the baseline; NGFW adds app, user and threat awareness; a WAF protects web apps specifically.
  • Default deny, ordered rules, egress filtering, and logging are the habits that matter.
  • Use host firewalls as well as network firewalls; place a DMZ between the internet and internal zones.
  • IDS detects, IPS blocks; NIDS sees the network, HIDS sees the host. Tune to balance false positives and negatives.
  • Encryption and fragmentation limit visibility, so combine network, host and log data.

Hands-on