Module 3 — Functional API: multiple inputs and branches

The previous module ended on four limitations of the sequential API. The functional API lifts all of them, at the cost of a single new idea: a layer is a callable object applied to a tensor.

The change of perspective

In sequential mode you declare a list of layers. In functional mode you wire tensors to one another and name the endpoints of the graph at the end.

from tensorflow import keras
from tensorflow.keras import layers

inputs = keras.Input(shape=(784,))
h = layers.Dense(128, activation="relu")(inputs)
h = layers.Dropout(0.2)(h)
outputs = layers.Dense(10, activation="softmax")(h)

model = keras.Model(inputs=inputs, outputs=outputs)

layers.Dense(128) creates the layer; the (inputs) that follows applies it. These two steps, merged in sequential mode, are separate here, and that separation is what makes everything else possible.

The tensor you manipulate is symbolic: it holds no data, only a shape and a dtype. Keras uses it to check layer compatibility at construction time, before any training.

Two inputs of different natures

Take a concrete case: predicting a property price from its photograph and its numeric features. Two kinds of data, so two processing paths before merging.

image = keras.Input(shape=(128, 128, 3), name="image")
table = keras.Input(shape=(12,), name="features")

v = layers.Conv2D(32, 3, activation="relu")(image)
v = layers.GlobalAveragePooling2D()(v)

t = layers.Dense(32, activation="relu")(table)

merged = layers.Concatenate()([v, t])
h = layers.Dense(64, activation="relu")(merged)
price = layers.Dense(1, name="price")(h)

model = keras.Model(inputs=[image, table], outputs=price)

Naming the inputs is not cosmetic: it lets you feed fit with a dictionary, far more readable and far less fragile than positional order.

model.fit({"image": images, "features": tables}, actual_prices, epochs=10)

Concatenate requires compatible dimensions

Concatenate merges on the last axis by default: every other dimension must match. Merging a (None, 8, 8, 32) tensor with a (None, 32) one fails. This is why the GlobalAveragePooling2D above is essential: it reduces the convolutional output to a vector, which is then compatible with the tabular branch.

Choosing a merge operation

Concatenate is not the only option, and the choice has consequences.

Merge layer	Effect	When to use it
Concatenate	juxtaposes, dimensions add up	branches carrying distinct information
Add	element-wise sum, identical shapes	residual connections
Multiply	element-wise product	gating and attention mechanisms
Average	mean	ensembling equivalent models

The distinction between Concatenate and Add is structural. Concatenation preserves all information from both branches and lets the next layer decide what to do with it, at the cost of a larger dimension. Addition superimposes the two signals and grows nothing, but assumes both branches live in the same representation space. That is exactly the residual connection mechanism from module 5 of course 07.

Two outputs, two losses, one weighting

A single network can learn several tasks, which is often beneficial: the tasks regularise each other.

inputs = keras.Input(shape=(64,))
trunk = layers.Dense(128, activation="relu")(inputs)

category = layers.Dense(5, activation="softmax", name="category")(trunk)
amount = layers.Dense(1, name="amount")(trunk)

model = keras.Model(inputs=inputs, outputs=[category, amount])

model.compile(
    optimizer="adam",
    loss={"category": "sparse_categorical_crossentropy", "amount": "mse"},
    loss_weights={"category": 1.0, "amount": 0.2},
    metrics={"category": ["accuracy"], "amount": ["mae"]},
)

loss_weights is the parameter people forget and must tune. The two losses have no reason to share an order of magnitude: a cross-entropy sits around 1, while a squared error on amounts in dollars can reach 10,000. Without weighting, the optimiser spends all its effort on the second and the classification learns nothing.

Setting the weighting without guesswork

Run one epoch first and record each loss's order of magnitude. Then choose weights so the contributions are comparable: if the regression is 500 times the classification, a weight of 0.002 on the regression brings them level. Adjust afterwards in favour of whichever task matters more to you — as a deliberate decision.

Sharing weights between two branches

A layer applied twice uses the same weights in both places. This is the foundation of architectures that compare two inputs.

encoder = layers.Dense(64, activation="relu")   # created once

left = keras.Input(shape=(128,))
right = keras.Input(shape=(128,))

el = encoder(left)     # same weights
er = encoder(right)    # same weights

gap = layers.Subtract()([el, er])
similar = layers.Dense(1, activation="sigmoid")(gap)

model = keras.Model(inputs=[left, right], outputs=similar)

Both inputs pass through the same encoder, therefore into the same representation space, and their difference becomes meaningful. Creating two separate Dense(64) layers would break the whole argument: each would learn its own projection and the subtraction would no longer mean anything.

Reusing a model as a layer

A keras.Model is itself callable, which lets you assemble blocks.

def residual_block(dimension):
    i = keras.Input(shape=(dimension,))
    h = layers.Dense(dimension, activation="relu")(i)
    h = layers.Dense(dimension)(h)
    o = layers.Add()([i, h])
    return keras.Model(i, o, name=f"residual_{dimension}")

block = residual_block(64)
x = block(x)   # slots in like an ordinary layer

This composition keeps deep architectures readable, and model.summary() then shows the block as a single line, which avoids three-hundred-line summaries.

Key takeaways

• In functional mode, creating a layer and applying it are two distinct acts; that separation is what enables branches, merges and shared weights.
• Named inputs let you feed fit with a dictionary; Concatenate preserves both branches' information, Add superimposes them without growing the dimension.
• With several outputs, loss_weights is essential: without it, the loss with the largest magnitude drowns out the others.
• Applying one layer to two tensors shares its weights, placing both inputs in a common representation space.

Next module: custom layers, for the cases where no existing layer does what you need.