How a CPU works: The bureaucratic point of view

Just like a bureaucrat a CPU is an unthinking thing that can only follow fixed rules, it can only accept certain instructions. Think of these instructions as forms that need to be filled in. In keeping with all red tape one form on its own is fairly useless but if you fill in enough of them you can eventually get results. A single CPU instruction can't do much on it's own but with enough of them you can do almost anything.

Think of a processor as a line of bureaucrats sitting at a long bench. Only these are a special type of bureaucrat, they do something useful. They do exactly the same work as the CPU in the computer you are using. A piece of paper arrives in the inbox at one end, the bureaucrat fills in a few boxes, puts the correct rubber stamp on the paper and then starts on the next form. Now in older computers that was the end of it, it only took one person to fill in a form. They worked their way through the pile of forms and got the work done. It was all a very simple and straight forward system.

Then some management consultant had an idea. Why not hire another bureaucrat so that we can get the work done quicker. But we couldn't have them both working out of the same inbox, it would be too complicated making sure they didn't both pick up the same form. However it is safe to split each form into two halves, the first person in line would fill in the top half and then pass the form on to the second person to fill in the rest. While that's happening the first bureaucrat in line could start on the next piece of paper. Since each person is only filling in half a form he can do it in half the time. While we're at it why stop there. Let's split the form in to lots of parts and hire a whole production line of bureaucrats.

Under this new system the first bureaucrat picks up the form, fills in his section and then waits, he can't pass the form on until the person next to him is ready. In order to make this pass over easier it is all timed using a buzzer. When the buzzer sounds he passes the paper on to the person next to him and starts on the next item in the inbox. The second bureaucrat fills in his section of the form, waits for the buzzer and then passes it on to the next person. This carries on down the line until the piece of paper reaches the end and has had every section filled in.

In processor terms the line of bureaucrats are the processor pipeline, the number of bureaucrats in the line is the number of stages in the pipeline. The delay between the buzzer sounding is the processor speed (1 MHz = 1 million buzzes per second).

Now marketing people will try to tell you that once piece of paper is processed every time the buzzer sounds and in a way that is true, most of the time one form is completed every buzzer. However for any individual piece of paper work it takes a lot longer for the work to be done, one buzzer per person in the line.

Under this system the obvious way of speeding up the paperwork would be to speed up the buzzer. The problem is that the pauses between the buzzer going off must be long enough for the slowest bureaucrat to do the most complicated task he can get, if he doesn't get time to do everything correctly then the whole system will break down. A simple solution to this is to split the longest task in two jobs and increases the number of bureaucrats. You need more staff but you can get more work done.

So what's wrong with this approach? Some of the time which forms need to be filled in will depend on the result of a previous form. The first person in your line may not be able to start on his next form until he knows a number that is calculated at the last stage of the previous form. The more people in your line the longer he has to sit around waiting. So do you just let everyone sit there and wait until you know what to do? No, you make a guess at what the result will be and start your first few bureaucrats working on the assumption that your guess is right. The downside is that if you guessed wrong you have to wait until all the incorrect paper work has got down the line and been thrown away before you can start on the correct paperwork. In CPU land this is called flushing the pipeline. The longer the line the bigger the delay this causes.

In processor terms this guess work is called branch prediction. It's normally very good (correct well over 95% of the time) but when it's wrong you have to wait a long time to get things fixed. If you have a very long line then this can be a huge penalty. Generally the longer the pipeline the more effort is put into improving the prediction.

So how else can you speed things up? Well you can combine several forms into one. Instead of trying to do lots of simple forms very fast you can do more complicated forms and spend more time with each person. The buzzer needs to go less frequently (or we need a far longer line) but since more work is done on each piece of paper the same amount of work is done in the end.

In a CPU the types of forms that are used relates to the rules that are followed, this rule book is known as the CPUs Instruction Set. A system with lots of simple forms that can be done quickly is called a RISC (Reduced Instruction Set Computer) design, a system with large number of complicated forms is called a CISC (Complex Instruction Set Computer) system.

RISC is called reduced instruction set because not only are the forms a lot simpler but there are less varieties of them, the down side being that you often need to fill in lots of forms to do one simple task. But at least you can do each one easily. RISC tends to use less power (simple forms take far less thinking to fill in) making it good for hand held devices. ARM and MIPS are both common RISC designs. Most desktop computers are CISC systems, they have lots of different forms that are very complicated to fill in. This makes them big and power hungry but they do get the job done. The Intel and AMD CPUs in windows PCs are called x86 processors, they use the instruction set first used on the Intel 8086 processor in the early 80's (with the addition of one or two extras).

An alternative way you can speed things up by having lots of separate lines of bureaucrats with one head bureaucrat who assigns work to each line. If we only had one or two bureaucrats in each line this would be very inefficient but as the lines have grown this becomes a more practical option. With several lines several pieces of paper can be done on each buzzer. The number of lines of bureaucrats is the number of pipelines in a processor. Often each line is dedicated to certain tasks, more highly trained and specialised bureaucrats can work faster than generalists who have to be able to do everything. The downside is that if you then get a job that requires skills that only one of your specialised lines has then a lot of people are sitting around idle and the effective rath at which work gets done drops significantly.

Another important factor is making sure that the inbox at the start of the line is never empty. In the good old days when you only had one person doing all the paperwork and popping off for a tea break every ten minutes this was easy. Now days with the rooms full of highly trained athletic bureaucrats the job has got a lot harder. The forms need to be fetched from a file room a long way away and which forms you need often depends on the results of previous forms. A lot of effort these days goes into making sure that the correct paperwork is available when the lines of bureaucrats needs it. If doesn't matter how fast your buzzer is going if everyone is just sitting there waiting for an arthritic old man with a zimmer frame to find the correct piece of paper. Lots of buzz words are used to describe tricks used to speed this all up. The speed of the old man is the memory speed, this combined with the number of files he can carry at one time gives the memory bandwidth.

In the old days the memory bandwidth was far more than was needed to keep the CPU busy. Files could be fetched far faster than they were ever needed. But now with modern bureaucrat training techniques the filing department can't keep up, they simply can't get the files fast enough.

One simple idea is to have several filing departments, no department is going to be able to handle all the files and get them quickly enough but we can create a far smaller, faster group. The line of bureaucrats only deals directly with this small, fast filing department. There is no way in which they can keep all the files handy but they can keep the most recent ones handy, they also predict which files will be needed in the future and fetch them ready for when they are needed. Unfortunately this prediction isn't always right and so every now and then our bureaucrats still have to stop and wait while something is fetched from the slow main filing room.

This system of having several levels of filing is known as Caching and it can all get very complex in an attempt to make sure that any delay is minimised. There are other buzzwords like memory bursts and cache line fills, these are both ways of getting other files that are near the ones you are using just in case you will need something similar soon. An alternative way of looking at it that you don't just get the one piece of paper asked for you bring the whole filing cabinet in case we need it.

So what next for the bureaucrats and they supporting departments. The basic ideas for how the lines work have just about been developed as far as they can with current technology, the length and number of lines will continue to grow slowly but it is reaching the point where adding more people just doesn't speed things up enough to be economical. So how do we get them to do even more work?

One idea that people have come up with and then almost given up on is known as hyper-threading. This is the cunning trick of giving each bureaucrat two desks, one on each side of them. They all work on one of the lines of desks until something goes wrong (e.g. they need to wait for something from central files). When this happens rather than just sitting around doing nothing they all spin around 180 degrees on their swivel chairs and start working on their second desk. To the outside world it looks as if there are two separate rooms of bureaucrats, each working at around half the speed that our one room is actually going, a 1 CPU computer looks like it has 2 CPUs. Unfortunately this idea isn't quite as good as it sounds at first. Lots of the applications out there just aren't designed to work well with two CPUs and the bureaucrats end up spending rather a lot of time spinning around (no they don't get dizzy). While a nice idea it just doesn't seem to work as well as people hoped.

Another "New Idea" is called 64 bit computing. This would translate to the boxes on the forms being twice as long as they currently are. But since all good bureaucrats use rubber stamps they can still fill in these longer boxes as quickly as they filled in the old, shorter versions. In other words we can do twice as much in the same amount of time. The problem with this is that firstly a lot of the time we just don't need that extra width on the forms and secondly most applications out there (including windows itself) are only available in the smaller form versions. It doesn't matter how wide the forms that we can fill in are if no one uses them. This will change but it will take time.

Finally people are now producing "dual core" and even "quad core" systems. Rather than the hyper-threading idea of having one line of bureaucrats pretend to be two lines we build a second room and actually have a second physical line. While this costs more in terms of wages it does, in theory at least, give us the ability to do things twice as fast. Unfortunately while this approach removes many of the disadvantages of hyper-threading it still falls into the problem that most things you would want to do on a computer just aren't designed to take advantage of the second CPU. You may have two rooms full of bureaucrats but a lot of the time one of them is going to be sitting around reading the newspaper or taking long lunch breaks.

Back to the front page.
xhtml css