Layered vs Pipes and Filters vs Clean Code

Some conventions for this blog.

• italics will often be used for technical words or their synonyms.
• Short code blocks will look like this

Definitions

What are Layered; Pipes and Filters; Clean Code Architectures?

Layered Architecture

Super simply: the system is architected into layers based upon processing data. Each layer processes the data and passes the processed data to the next layer. Layered is an architecture which once implemented will be hard to change; dependencies flow down to the next level.

Clean Architecture

Clean Architecture is similar to Layered, the layers are different. Think of the layers as concentric circles and the dependencies flow towards the center. Clean Architecture dependency flow is the prime differentiator.

Pipes and Filters Pattern

Again, very simply, the system is architected into modules call filters which are connected with pipes. More of a pattern than an architecture, but how the pipes are implemented is a part of the architecture.

For example you can wire filters using queues like SMS or RabbitMQ, topics like SNS, or have filters pass the data onto the next filter like in the Chain of Responsibility pattern. The latter is highly discouraged and technically not a pipes and filters implementation. The value in adding complexity like pipes and filters is in reusing filters in other workflows.

You could use something like delegation to configure which filter a filter forwards its results to. You'll be left implementing your own retry and error handling where SMS and SNS have a lot of retry and error handling built in.

class ParseInputFilter implements Filter {
    private Filter targetFilter;

    public ParseInputFilter(Filter target) {
        targetFilter = target
    }

    Output filter(Data data) {
        // filter data
    }
}

Huge Downside, besides implementing retry and error handling logic, is rewiring filters requires code changes. I strongly recommend you do not wire filters this way.

A popular and much used implementation of pipes and filters is *NIX commands. Most shells have the pipe | that pipes the output of one command into another command.

grep -lr Flags:\ draft | grep \.md

Returns how many of my Markdown blogs are drafts.

Differences

Pipes and Filters is a pattern that can be implemented within an architecture. Layers Architecture and Clean Architecture are architectures.

Layers Architecture and Clean Architecture have different dependency models. Layers Architecture dependencies always flow down to the next layer. Clean Architecture dependencies always flow towards the middle with details in the outer ring and business and domain logic at or near the center.

When to Use

When should I use each architecture? TLDR; Use Clean Architecture with Pipes and Filters for data processing.

Layered

Why not use Layered Architecture; isn't it the most used architecture is software development? It may be, but most software projects fail. We've gotten a lot better since the early days when that number was near 90%, but we're still behind the curve. I'm not saying there's a causation, but there's certainly a correlation.

Layered architectures are hard to maintain and enforce. At some point a developer is going to copy/paste code between layers because he needs the same processing. Bugs in upper layers can often bring down the whole system.

The layers cause dependency issues or extra layers just to keep things clean. For example your business layer may depend directly on the data store layer. Your business logic should never know how the data is stored. The solution is often add another layer called repository between the business logic and the data store that provides an API that closely matches the domain language.

As Uncle Bob says, "Your business logic should never depend on the details". Details are frameworks and drivers: UI, database etc.

So every layer that needs to use a detail will need a layer between it and the next layer to hide the details from it.

Pipes and Filters

Use the Pipes and Filters pattern when either you need to reuse filters in other chains.

For example, you probably want authentication and authorization filters in all chains.

Don't start with it. Keep your architecture simple and pay attention to the logic streams. When you start needing the same logic in multiple streams then consider refactoring to a Pipes and Filters pattern.

Clean Architecture

This should be your end architecture unless you're building a small and simple app. Then Layers and Dependency Inversion will most likely be enough to keep your architecture clean and maintainable.

There you have it. My take on three ways to organize your code. The best thing to do is keep it simple and clean. Rely on the SOLID principles to guide you.

When you continuously refactor to SOLD principles, what architecture you fuse or what you decide to call it doesn't matter. Your code will be clean and maintainable and that's the goal.