LLMs for Engineers — Part 5

So far in this series, we’ve broken things down step by step. We started with where the data comes from, then saw how text becomes tokens, and then understood how the model predicts the next token again and again.

At this point, you already know how the system behaves from the outside. But there is still one big missing piece .What is actually inside the model that makes all of this possible?

If we simplify everything, we can think of the model as a function. You give it a sequence of tokens, and it gives you probabilities for the next token. That’s the behavior we discussed earlier. But that function is not magic. It has a structure, and that structure is what we call a transformer.

Before transformers came into the picture, models used to process text step by step. They would read one word, then the next, and then the next. This worked, but it had clear limitations. It was slow, and more importantly, it struggled to handle long sequences. If something important appeared earlier in a sentence, the model would often lose track of it by the time it reached the end.

Transformers changed this completely by introducing a different way of looking at text. Instead of processing words one by one, they look at the entire sequence at the same time. This might sound simple, but it is a big shift. It means the model doesn’t have to “remember” things in the same way. It can directly look at any part of the input whenever it needs to.

To make this more concrete, think about a sentence like “the router that connects to the core switch is down”. When you read this, you naturally understand that “is down” refers to the router, not the switch. You connect different parts of the sentence without even thinking about it. The transformer is designed to do something similar, but using numbers.

This is where the idea of attention comes in. Attention is the core mechanism inside a transformer. In simple terms, it means that for every token in the sequence, the model looks at all the other tokens and decides which ones are important. Not all tokens contribute equally. Some carry more useful information for the current prediction, and the model learns to focus on those.

You can think of it like this. Each token is asking a question: “which other tokens should I pay attention to right now?” The model then assigns weights to all the tokens based on their importance. Tokens that matter more get higher weights, and those that matter less get lower weights. This creates a web of relationships across the entire sequence.

Once these relationships are established, the information flows through multiple layers inside the transformer. Each layer refines the representation a little more. At the beginning, the model might only capture very basic patterns. As we go deeper into the layers, it starts capturing more complex relationships and context. By the time we reach the final layers, the model has built a rich internal representation of the input sequence.

From there, the process is similar to what we discussed in the previous post. The model takes this refined representation and produces probabilities for the next token. So even though the core task is still next-token prediction, the transformer makes that prediction much more informed by understanding how tokens relate to each other.

One way to think about the transformer is that it allows every token to interact with every other token. Instead of a straight line where information flows from left to right, it creates a fully connected view of the sequence. This is why it works so well for language, where meaning often depends on relationships between words that may be far apart.

At this point, it’s important to remember something we’ve said before. Even with all this complexity, the model is not actually “understanding” language the way humans do. It is still working with numbers and patterns. What the transformer gives us is a much better way to capture those patterns, especially when context matters.

If you connect this back to the earlier posts, the full picture becomes clearer. We start with tokens, feed them into the transformer, use attention and layers to build relationships, and finally produce probabilities for the next token. That output is then sampled and added back into the sequence, and the process repeats.

So while the behavior of ChatGPT might feel complex from the outside, the core idea remains consistent. It is still predicting the next token, one step at a time. The transformer is simply the reason those predictions are so good.

Simple network architecture, the Transformer Link for reference

https://arxiv.org/abs/1706.03762

Let’s unpack this in plain language.

Not Understanding — Predicting

When most people talk about AI — especially generative models — they assume the system understands like a human does. That’s not true.

Generative AI is trained on huge amounts of data and then uses statistical patterns to generate new content. It’s not thinking. It’s not reasoning. It is incredibly good at predicting what comes next based on patterns it has seen before.

If you ask it to write an essay, it doesn’t sit down and think about ideas — it predicts word patterns that make sense based on training data. If you ask it to generate an image, it doesn’t imagine like a person — it stitches together elements that look right together in the data it saw during training.

Generative AI in Practice: A Different Kind of Collaboration
So what is it really good at?

It can produce new artifacts — text, music, images — that look original.But this “originality” is really a recombination of patterns it has already seen. It’s creativity without consciousness, intuition, or intent.

So Why All the Hype?

Because humans are pattern-seekers too. We look at outputs that seem cohesive and pretend the machine has internal understanding. That’s not just inaccurate — it’s dangerous.

Hype focuses on what AI looks like it can do, not what it actually can do.

This leads us to two common pitfalls:

❌ Believing AI is a thinking partner

AI doesn’t think, it predicts.

❌ Assuming it’s always correct

Pattern prediction can still be flawed , biased or incorrect ,especially if the training data is poor or incomplete.

What Really Matters

Here’s the insight most people miss:

Generative AI is a mirror — not a mind.
It reflects patterns from provided data ,not understanding from experiance.

This distinction matters because it defines how we should use it:

It’s great for augmentation -summarizing, drafting, pattern spotting.

It’s not a substitute for expert judgment or contextual reasoning.

As professionals - whether in tech, products, analysis, or decision-making — this should be our lens:
We don’t ask what AI can generate.
We ask what patterns we should trust, and why.

Final Thought

The beauty and the risk of generative AI aren’t in what it outputs.
They’re in how people interpret those outputs.

If you chase the pattern - the shiny result-you’ll be distracted by noise.
If you chase the meaning beneath the pattern - that’s signal.

Subscribe now

Let’s Bring It Into Networking

This is where it gets interesting for us.

Say you’re managing a large network — multiple sites, lots of routers, switches, logs , lot of support tickets.

Now imagine Gen AI trained on your own environment — logs, past tickets, config files, SNMP data, etc.

Here’s what it can do.

Example: Automatic RCA Report with Gen AI

Problem: You see packet loss in Region X. Alerts come in.
Old Way: You log in, check logs, dig through past incidents, prepare RCA. Hours gone.

With Gen AI:
You just type below prompt:

"Generate RCA for packet loss on Router R2 at 4PM in Region X"

And it replies:

"Packet loss caused due to MTU mismatch post-software upgrade on R2. Similar problem on 14 Mar & 3 June. Recommend audit + MTU compliance check."

Straight to the point.

⏱️ Time saved: Big
📄 Consistency: High
📚 Backed by actual data: Yes

There are so many other case too , like config generator ,capacity manangment ,audit repot etc.

Why It Works for Us

• It handles logs, tickets, config — all the messy stuff

• It finds patterns we may miss

• It can explain things, not just detect

• It keeps getting better with feedback

Final Words

Generative AI is like a team member that doesn’t sleep. You give it data, it gives you solutions.
You ask for help, it explains. It’s not perfect — but it’s fast, useful, and always learning.So don’t think of it as a threat.
Use it as a tool — to save time, reduce noise, and work smarter.

You have got big warehouse, drivers and a road connecting to all destinations.

🧠 Compute: The Brains — or the Drivers on the Road

Compute layer is like team of drivers ground around, moving packages (data) from Point A to Point Z.

CPU are all purpose vans -- they can handle anything but aren't used for speed delivery.

GPU are the specialized trucks that carry huge volume fast -- Built for heavy duty tasks lets say like training neural network

RAM That is essential part that every driver need like dashbaoard in front of hime

NICs Think of its like high-speed lanes of tunnels that helps you bypass traffic and get to destination faster

Why the love with GPUs lately? Because AI training isn’t normal — it’s like moving large chunk of data around.

GPUs are wired to handle that kind of workload efficiently, with thousands of cores working in parallel.

Some systems stack up 8, 16, even more GPUs in a single server — all linked with ultra-fast NICs pushing speeds up to 800 Gbps.

That’s not a typo. We're talking about transferring entire libraries worth of data in the blink of an eye.

📦 Storage: Your Warehouse System

Now, where do all those packages come from? Right — the warehouse.

In AI, storage plays that role. But it’s not just about capacity. It’s about speed. If your drivers are sitting around waiting for boxes to load, your whole system slows down.

SSDs are the fast-loading bays — perfect for handling large datasets quickly.

InfiniBand with RDMA? It’s like teleporting data straight to the truck, skipping the harder task. (in our case the CPU). Again this itself is very big topic and will love to come with post specfically to InfiniBand with RDMA.

IP-based storage is more plug-and-play. Maybe not the fastest, but it works well with what most systems already have.

And distributed storage spreads data out, so there's no single point of failure. It's like having multiple warehouses stocked with the same inventory — so if one goes down, another picks it up.

Subscribe now

🌐 Networking: The Road System That Connects It All

Networking is the glue. Or, sticking everything together, it's the actual road system your fleet depends on.

Most AI data centers use something called spine-leaf architecture:

Leaf switches connect directly to compute nodes — the endpoints.

Spine switches tie everything together, making sure data can flow across the entire network without bottlenecks.

If that sounds technical, think of it like this: it's a carefully designed freeway system with enough lanes to handle rush hour — or in this case, petabytes of data moving in real time.

And to keep things moving? Engineers tweak the subscription ratio — the balance between how many compute nodes share bandwidth and how much is actually available. Get it wrong, and things slow down. Get it right, and data moves so efficiently it barely feels like it’s traveling at all.

🔧 When Everything Clicks

Put all of this together — compute that’s optimized, storage that’s fast and accessible, and networking that keeps things flowing — and you’ve got an AI infrastructure that can scale to almost anything.

That’s what powers:

It’s not about any one component. It’s how they all work together.

Smiles :)

Anurudh

In this post , we will talk about “How AI Learns”

• Supervised Learning :How we train machine with example with correct answer

• Un Supervised Learning: How Machine finds hidden pattern when provided with unlabeled data

• Reinforcement Learning :How machine leans by trying things out using feedback

🔍 Supervised Learning :Teaching Machine With Example

Its basically to teach a machine using examples that already have a answers. So we teach model with real example with both input and the correct answers. Over the time ,it starts to get in.

Supervised learning is like teaching someone with flashcard.

Imagine this:

You are sitting with kid showing him flash card of shapes. Each flashcard has a label:

“Square” “Triangle” ” Circle”

Keep showing him flash card over and over.

After sometime , show him flash card having new shape without any label and ask:

“What is this ?

Kid will reply with correct answer may be “Triangle”

That’s supervised learning—you learn from examples that already have answers (labels), and then you try to guess the answer for new stuff.

🎯 Real-Life Example – Predicting House Prices

Okay, Let’s say we are building a model to guess house prices. we provide relevant input to it:

• How big the house is

• Where it’s located

• Number of rooms

...plus what it actually sold for. We keep giving it example after example, and slowly it starts to notice, “Hmm, houses in Mumbai facing Sea go for more. “

Now, if you hand it data for a brand-new house, it can take a decent guess at the price—even if it’s never seen that specific one before.

Subscribe now

🌐 How This Works in Networking

Lets consider —network data—we might use this to catch bad traffic.

we train the model on labeled flow data. Stuff like:

• IP addresses

• Ports

• Traffic volume

• Protocol used

• Packet size

• Time of day

• Duration

And for each one, mark it “Safe” or “Malicious.”

We do that long enough, and the model starts spotting patterns. “Ah, this combination of port + packet timing? Seen that before. Pretty sure it’s a X”

This is where it starts getting useful—real-time detection, with way less manual effort.

So Why’s It Called “Supervised”?

Simple: because We are literally supervising the learning process. Like showing kid how to ride a bike. We are e right there holding the seat, giving feedback as they go.

It’s not figuring stuff out on its own yet—We are guiding it.

🔍 Unsupervised Learning: When There Are No Answers

So, this one’s bit different. There’s no training, no right answers, nothing. basically give the model a bunch of raw data and say, “Here. Figure it out.”

It’s like landing in a city you’ve never been to—with no map—and just walking around. Over time, you notice stuff: “Okay, everyone gathers near that bus stop,” or “This street’s always jammed during peek hours at 6 PM.” That’s what unsupervised learning does. Finds patterns. Groups.

🎯 Real-life Example – Retirement Savings

Take this—say you’ve got a dataset of people’s ages and their savings. That’s it. No idea who’s doing well, who’s behind. Just raw numbers.

Now, the model might look at it and say, “Hmm, looks like these over-50 folks with low savings—yeah, one group. Then these younger ones saving aggressively? Another group.”

No labels. No judgment. Just clusters. Then a human (like a financial advisor) can come in, look at those clusters, and go, “Yeah, that first group’s in trouble. These others? They’re good.”

Share Automation

🌐 Network Example – Spotting Behavior

Okay, now let’s bring it to networking.

Say you’ve got logs—tons of unlabeled flow data. You dump it into a model. It’s got no clue what “normal” even means, but still starts spotting patterns.

Maybe it figures out that from 9 to 5, traffic looks pretty normal. But then at 2 AM, suddenly some host is pushing massive files to some unknown IP. The model’s like, “That’s not the usual routine.”

Boom. Anomaly detection.

That’s the magic. No one tells it what’s bad—it just sees something off.

🎮 Reinforcement Learning: Learning by Doing

This one’s honestly a gaming. Feels like how we all actually learn.

Reinforcement learning is trial and error, plain and simple. The model does something, sees what happens, and adjusts.

Do something good? Get a reward.
Mess up or wrongly executed ? Penalty.

🎯 Example – Teaching a Game Bot

You’ve got a bot learning to play a game. You don’t tell it the rules. You just give it a little reward when it scores.

You just say,
🟢 “Score point? +1 reward”
🔴 “Lose a life? -1 penalty”

First few rounds? It’s clueless. But it keeps trying. Gets a little better. Tries different moves. Slowly starts winning more.

Eventually it discovers the strategy to win.

Same as how a kid learns. Fall off the cycle? Stay upright?

🌐 Network Example – Auto-Tuning Without the Manual Work

Now, think networking.

Imagine a system that’s allowed to adjust routing paths, bandwidth, QoS settings—whatever.

It tries something new. If latency improves, or throughput increases—reward.
If things get worse—penalty.

And over time? It learns the best combo. Without a human manually tweaking BGP attribute or adjusting buffer size

It’s literally self-tuning. Just like we all do after messing up a few times and figuring things out.

We will talk more about AI in coming posts.

Till time , keep learning !!!!

Smiles :)

Anurudh

Have you wondered how computer can recognize face , talk like human , suggest for next movie or even suggest what you need to buy . Let's explore this transformation together.

The Early Days: Traditional AI

A long time ago , Scientist wanted computer to think like people. This idea was called Traditional AI. Because of their work, computers got better at solving problems and understanding information.

One smart person, John McCarthy, made a special computer language called Lisp to help with AI. Lisp is really good at working with symbols, like letters and numbers. Even today, some scientists still use it when working on AI.

Thanks to Traditional AI, we also learned more about how the human brain works and how computers can copy some of that thinking.

Subscribe now

But Traditional AI had some trouble. It followed very strict rules and didn’t do well with messy or confusing problems. So, people came up with better ways to make AI smarter — like machine learning and deep learning. These new methods help computers learn from a lot of data instead of just following rules.

That’s how AI keeps getting better and better over time!

The Shift: Embracing ML and DL

Today, there are two main types of smart computer programs: Machine Learning (ML) and Deep Learning (DL). They both learn from data, but they do it in different ways.

It's important for people who build and use these programs to know the difference between ML and DL. This helps them pick the best one for what they need.

Share

Knowing about both also helps people come up with new and better ideas. Sometimes, they even mix ML and DL together to get the best of both! For example, they might use ML first to clean up and organize the data, and then use DL to find really tricky patterns and make smart guesses.

Machine Learning (ML)

Machine Learning (ML) is when computers use different algo to get better at tasks the more they practice.

In simple words, ML is like teaching a computer by giving it lots of examples. After learning from these examples, the computer can guess or figure out things it has never seen before!

A good example is how email spam filters work:

The computer looks at tons of old emails that were marked as "spam" and "not spam," learns the difference, and then tries to catch new spam emails all by itself.

Deep Learning (DL)

Deep Learning (DL) is a special kind of Machine Learning.

It uses something called artificial neural networks, which are a lot like the way our brains work. These networks have many layers that help computers learn from huge amounts of information.

Deep Learning can look at things like pictures, words, or sounds and figure out patterns.

For example, it can help a computer look at a photo and say what’s in it!

Because DL uses so much data and many smart layers, it can be very, very good at things like recognizing faces in pictures or understanding what someone says in a recording.

Data Needs for ML and DL

Machine Learning (ML) doesn’t need a lot of data to work well.

Imagine you want to teach a computer to guess house prices — you only need a small list with basic information like house size and number of rooms.

Since it doesn’t need tons of examples, it’s faster to teach and doesn’t need super powerful computers.

ML is also easier to understand — you can usually figure out why it made a certain choice.

That’s why ML is great when you don’t have much information or when the information is nice and neat (like tables).

But ML struggles with messy stuff like pictures or sounds.

Deep Learning (DL) is different — it needs way more data, especially messy things like photos, music, or books.

Even if the information looks small (like a short story), DL still takes a lot of time because it has to work really hard to understand it.

Teaching a DL computer to recognize voices, for example, could take days or even weeks!

It also needs big, powerful computers to do all that work.

And, it's much harder to understand how DL makes decisions because it’s like a giant maze inside the computer.

We will talk more about AI in coming posts.

Smiles :)

Anurudh

Buy Me a Coffee

Traditional AI = Knowledge Base + Inference Engine

Before whole Buzz around CHatGPT and Generative AI .The AI was was built on below 2 parts:

Knowledge Base : Collection of domain specific knowledge and rules

Inference Engine: Its logical mind that takes rules define in knowledge base to reach conclusions and decisions step by step .

How did it work?

Traditional AI flow will begin with user input where user will ask questions .

System wouldn’t guess any answer rather will refer all respective domain knowledge and rules are already placed in knowledge base represented symbolically and logically.

Thereafter rules and and data are retrieved from knowledge base by inference engine and try to make sense of it using the rules defined.

It doesn’t guess ones , It keep checking rules again and again to become better each time till it finds good answer.

This is something like solving a big puzzle .End result is answer provided by system based on user input and whatever system learned from the rules.

Quick Example:

Lets understand with an example :

Knowledge Base:

• Rain → Raincoat

• Cold → Jacket

• Sunny and warm —> Tshirt

User Input — > What should I wear its cold and rainy outside

Inference Engine:

• System read user input and looks at rule defined in knowledge base

• Breaks down as user input into “Cold” and “Rainy”

• Matches the rules which it knows and start thinking step by step

• Cold =Jacket ? Rainy = Raincoat ?

• It has to come out with better choice.

• System replies to user” You should wear a waterproof Jacket”

Thus system didn’t gave any random output but used facts and rules defined in knowledge base and thought it over step by step and came with best output .

Buy Me A Coffee

So traditional AI is slow careful but not creative .

We will talk about Traditional AI challenge in next post .

Smiles :)

Anurudh