Math: The Stats Ma'am... Just the Stats

Look... I know some people don't like math. I know some people struggled with it in school and don't see the point of advanced math. I also know I was one of those kids who really liked math and to whom it all made sense. Until I got derailed early in university (but I digress).

One branch of mathematics is rising to a new found prominence - statistics. Statistics is incredibly powerful for two different reasons.

First it allows us interpret information about large groups (people, cars, planets, etc) by looking at a small sample of the things we're looking at. You don't have to ask everyone in a country how they'll vote to run an opinion poll - though I do still think we should all get our individual votes counted in the election. Statistics tells you how to find the group properly (so as not to end up with a biased result), tells you how to ask the questions, and tells you how to interpret the results. It even can tell you how accurate the results of the sampling are.

Secondly statistics allows you to understand what's behind numbers. How to pluck information and meaning out of piles and piles of information. With computers we have more information than every. In many cases businesses and governments don't have to worry about sampling the data since the computers can look at it all. Learning about what the information can tell us is very important. Especially when there is so much information going around.

If you want a light hearted but serious look at the power and uses of statistics then you should look no further than Hans Rosling's program on the BBC The Joy of Stats. Hans Rosling is a professor of International Health and Director of the Gapminder Foundation - which is hosting the Joy of Stats and other videos. Hans burst onto the scene with several celebrated TED talks. He's a fantastic public speaker. The same page that holds the Joy of Stats also links to some of his other videos.

Poke around and see how modern stats can change how you think. Hans Rosling will make you reconsider the idea of the third world. He'll make you change your mind on population growth and give you new insights into HIV and other diseases. He may even make you think that statistics is sexy.

Math: Public Secrets?

We tend to use a great deal of complex mathematics everyday. Or more correctly we tend to use technology that uses complex mathematics everyday. One such complex area is cryptography. When we log on to our bank account or our online email there's a lot of math going back and forth.

This isn't a post about how cryptography works. It isn't really a post about details of math and mathematical processes. Instead it's a post about unsung and unknown computer scientists who came up with the idea we now call public key cryptography and who haven't, until recently, received any recognition for their work.

The Alternative History of Public-Key Cryptography looks at one of the fundamental algorithms of our time. Whether or not you know it or understand it you use it daily. Like many of the ideas in the world of cryptography it was first conceived of in secret. Later it was rediscovered and brought to the world at large. Now it's time to give credit where credit is due.

Math: A Classic Reborn

All the talk of the long S last week reminded me of a classic math book. Actually it's a classic version of a classic text.

Euclid's Elements is a 13 book classic on geometry and math. In 1847 Oliver Byrne created a version of the first 6 books in which he used colours to indicate the different elements in a drawing instead of using letters and other labels. A scanned public domain version of the book is available thanks to the Mathematics Department of the University of British Columbia.

It may take a few minutes to get to used to the long Ss but after that... it's relatively smooth sailing. Now why didn't texts like this end up as part of my education?

Math: Is Math Misunderstood?

Dr. Robert H Lewis at Fordham University wrote The Misunderstood Subject which is about how math is perceived by most people.

He makes a great case that math is misunderstood. It's not the formulas, it's not the exact steps, it's the process that's important. He gives several parables to make his point. He gives a good overview of how training and education differ.

What he doesn't do is continue the argument. He doesn't show what mathematics can do. Oh he talks briefly about computers but he only gives a technical example.

To my mind the essay is half complete. Give me the rest. Show me how the process of mathematics can help. Show me how it's applied. Show me how it helps. Then we won't just be defending the need for math education. Then we could be inspiring people to learn math.

Math: Representing Numbers

Mathematics is formulas and theorems, concepts and numbers. In today's world mathematics is also in computers. That is where the rubber hits the road.

Putting numbers inside of computers isn't as easy as it sounds. Putting decimal numbers into computers is fraught with problems.

Should you ever find yourself programming computers in a way where the computers are manipulating numbers for you please sit down and read What Every Computer Scientist Should Know About Floating-Point Arithmetic.

It's not as easy or as simple as you think. Let David Goldberg tell you about many of the pitfalls and problems in getting computers to represent numbers.

Math: At Which Point Does it Get Absurd?

In the ongoing game of "my computer is bigger and faster than yours" there seems to be one particular calculation that helps determine the winner of the game.

To calculate as many digits of pi as possible. Now I've mentioned pi before but not as something to calculate. One can never calculate pi exactly since it is a transcendental number. The decimal places keep going and going without repeating.

Which makes it a perfect number to calculate on computers because you can keep calculating it for as long as you like. There is no built in limit to how many digits you can compute.

Computing pi by hand is tedious and error prone. Nevertheless over time people sat down and did the calculations. Starting in 1947 or so computers also cranked out digits. Here is a short chronology of the number of digits calculated over time. This list ends in 1999 with a computer calculating 206 billions digits.

Eleven years later that number has been surpassed and not by a little bit. The current record for the number of calculated digits of pi is.... 5 trillion. That's 5,000 billions or 5 million million digits -(5,000,000,000,000).

Better yet it wasn't calculated on a supercomputer nor on a cluster of specialized machines nor on a specially built computer. It was calculated on a desktop computer in 90 days.

Granted it was a fully loaded desktop computer. The specs are within the reach of anyone with lots of cash and access to a computer store:

• 2 Intel Xeon processors (12 cores total)
• 96 GB of memory
• 19 x 2 TB disks and 1 x 1 TB disk

And it ran Windows Server. Given similar (or better) hardware and a bit more time maybe you an calculate a new record yourself.

The story of the calculation, with details, some pictures, and access to the program that did the calculation is all here.

After years of these types of calculations being done on supercomputers and specialized machines it's interesting to note that home computers have progressed this far. I wonder who's taken up the challenge to calculate a few extra trillion digits? I wonder if somewhere there's another high end typical computer cranking away to set a new record.

Math: Fun With Serial Numbers

Look around you. How many things near you have serial numbers? Almost anything technical or technological has a serial number. We take it for granted that many manufactured items have been numbered.

What can you do with those numbers? One story tells of how the allies in WWII were able to estimate the production of German tanks by collecting the serial numbers off of destroyed or captured tanks. There are a few variations on the story. Each has slightly different numbers in involved. In 2006 Gavyn Davies explained the math in the Guardian. If Wikipedia is to be believed there was additional math involved that even looked at an estimate of the number of molds the Germans had to make tank wheels.

What about using serial numbers in the present day? Is there information hidden in modern serial numbers? Could you get enough serial numbers to uncover this information?

The answers are yes, yes, and yes. Thanks to the internet it is easy to ask lots of people to type in serial numbers. Once there are enough of them... the investigation begins. Here's what can be found out from the serial numbers from a large number of Microsoft XBoxes.

Math: The Games People Play

There are quite a few branches of mathematics. One that has caught my interest on more than one occasion is game theory. Game theory tries to understand how and why people react in certain circumstances. It's about strategic thinking. It's about games we play every day.

Why do we decide to do what we do? How do we judge what is the best course of action? It sounds a bit vague and hard to capture in equations but it is a branch of mathematics.

A Chronology Game Theory has been put together by Paul Walker. It starts with game theory in the Talmud and moves on from there. Mainly it's a collected bibliography but it gives a quick overview of the subjects covered by game theory.

A more detailed work is Roger McCain's Strategy and Conflict: An Introductory Sketch of Game Theory. This started as a work from which to teach an undergraduate course in game theory and eventually ended up rewritten as a complete book. Even before the rewrite it stands as a good guide to the ideas and principles of game theory.

Pretty soon you'll be up to speed on the Prisoner's Dilemma and Tit-for-Tat. At which point you'll either be a better strategist in the areas where game theory is helpful or you'll be even more confused than before. I'm not responsible for what happens.

Math: Who Needs the Exact Answer?

The point of knowing mathematics is to get the answer to problems right? Preferably the correct answer. Otherwise what's the point?

Mathematics isn't just an end in itself. It isn't something to learn so you know math. At its most useful mathematics is a tool that lets us do other things. It's a tool that gives us insight into the world around us. Learning to use math as a tool not only makes it more practical but much more useful.

Which is why Sanjoy Mahajan's book Street-Fighting Mathematics: The Art of Educated Guessing and Opportunistic Problem Solving (Creative Commons pdf) is such a great read. Working through various approaches to help make informed guesstimates and improve your mathematical problem solving skills he makes practical math skills much less intimidating. If you want a very practical book to help you improve how you use math, instead of just how to improve you math skills, then I suggest Street-Fighting Mathematics.

Math: The Power of Video

I've talked about the power of video in education before. It's amazing how watching a good teacher can made things seem so much easier. Here's a couple of example dealing with calculus.

Let me start with the absurd. All of Calculus in 20 minutes. Yes it's a promotional video for a calculus course but don't let that bother you too much. I'm certainly not endorsing this as the way to embrace powerful new ideas. Still it's fun to watch so much crammed into a few minutes.

A much more useful introduction to the ideas behind calculus is from MIT's Open Courseware. Professor Gilbert Strang put together a few short videos to introduce the big picture behind calculus. There is also an entire calculus textbook he wrote available.

Unlike calculus in 20 minutes that makes things almost impossible to follow I find that Professor Strang makes it seem that it will be easy to learn. Which is all I ever asked for from a teacher.

Math: It's Not Important... It's Just Math Right?

Mathematics may be the most pure of the sciences. Formulas, notions, suppositions, and conclusions. It seems so abstract and not very real.

After all algebra has its letters and geometry has its points, lines, and circles. All abstract constructs. But math can be very applicable to the real world. Correct and accurate math is even more applicable.

Take the idea of insurance. I'll pick fire insurance as an example. The math is really simple. If you have 1,000 homeowners paying in to fire insurance policies then you need to make sure that you get in more than you will have to pay out. How do you determine how much you'll have to pay out? First figure out how much each fire claim will cost you. Then figure out how many claims you'll get per year for those 1,000 homeowners. Put the two together, work in some extra to run your business and some extra for profit and voila... you can run an insurance company.

Of course real life is more complicated than the simple math would indicate. The type of house, the size, the construction, the neighbourhood, the wiring, the fire alarms and smoke detectors, and any number of other variables make some houses more risky than others. As well there are average years with a certain number of claims but then there are exceptional years. Some with fewer claims and some with many more claims than is typical. You have to be able to pay up when an entire housing division goes up in smoke.

Getting the math right is crucial. Getting the complex, convoluted, and well thought out math right is even harder. If you're insurance company manages to get it right you'll end up covered during disasters instead of watching your insurance company go bankrupt and not be able to pay out claims. Catastrophes can cause problems for insurance companies and not just the immediate victims.

The Acts of God Algorithm by Art Jahnke tells the story of a new better way of looking at risk. Especially catastrophic risk. It's amazing what happens when someone looks at an old problem in a new light.

Math: Benoit Mandelbrot

Not many people can say they've added a word to the language of the world. Not many of us can create a word that sticks. Certainly not many can come up with a word that somehow captures complicated subjects and makes them seem accessible. Benoit Mandelbrot did that with the word fractal.

The word invokes complex and wondrous pictures. Colours exploding on computer screens and created by simple mathematics applied with the repetitive power of computers. The word hints at how the images are self similar at different scales - you can see smaller (or fractional) versions of larger objects as you zoom in. It's a word that sounds good and works well.

Benoit Mandelbrot passed away at the age of 85 ten days ago. His legacy in the world of math and finance is an impressive one. But to many of us he will be remembered as the person who gave the world fractals.

Let him tell you about it in his own words.

Math: Learning to Read

Every discipline has its own way of presenting ideas to the world. In science, for example, the ideas are presented in a form that is so well known it can be parodied. (I can still crack certain people up just by saying "chicken chicken chicken").

Math has it's own format and structure for presenting ideas. Knowing that there is a structure and an approach is one thing. Being able to make sense of it and work your way through math is another. Luckily Shai Simonson and Fernando Gouvea have written a primer to help that's simple titled How to Read Mathematics. Not only does the authors offer advice but they walk you through understanding a few simple math problems to see how you one understands math writing with practical examples. It's well worth the time and effort to wrap your mind around.

Math: Standup Numerology?

What do you get when you take an inventive offbeat mind and let it loose on numbers? What happens if that minds is inside a standup comedian? You get Charles Fleischer's theory of Moleeds.

Recently people were recently made aware of Moleeds thanks to one of the most offbeat Ted talks ever.

Some of us have encountered Moleeds before. After all Charles Fleischer has been going on about his theory for a long time. Here he is in 1990 explaining Moleeds in more detail. In case you want to follow along or study his ideas there is an online transcript of his older routine.

Math: The Most Important Video You'll Ever See?

The catchy title on YouTube is The Most IMPORTANT Video You'll Ever See. The title in the video itself is Arithmetic, Population, and Energy. Regardless of what you think of the catchy title the video makes a powerful point.

Dr. Albert Bartlett from the University of Colorado at Boulder sits behind a desk, shows some slides and charts, and will probably scare the heck out of you. If you don't have a handle on the idea of exponential growth or compound interest this will be an eye opener.

The video is in 8 parts. Here are the links - part 1, part 2, part 3, part 4, part 5, part 6, part 7, part 8.

I'm not sure it's the most IMPORTANT video but I'd recommend it even if you aren't fluent in math. It will make you think a bit about our future.

Math: Mathematics Humour - Alice and Bob

In 1978 Ron Rivest, Adi Shamir, and Len Adlemen (known as RSA) published a paper entitled A Method for Obtaining Digital Signatures and Public-Key Cryptosystems (pdf). All modern public key cryptography descends from the work in this paper. If you use PGP or GPG then you are using math that was pioneered by Rivest, Shamir, and Adlemen.

The paper describes how the system works by working through scenarios where A and B are trying to communicate securely. Ron Rivest has said that he came up with the names Alice and Bob so he could use A and B in mathematical notation. Having one male and one female also meant that "he" and "she" could be used in the scenarios and people would know which "person" the authors were talking about. The introduction of the most famous couple in cryptography is as follows:

For our scenarios we suppose that A and B (also known as Alice and Bob) are two users of a public-key cryptosystem. We will distinguish their encryption and decryption procedures with subscripts: EA,DA,EB,DB.

Now Alice and Bob were soon joined by others. Eve is an eavesdropper on the conversations between Alice and Bob. Mallory is a malicious attacker. The list goes on and on.

Only 6 years after the original RSA cryptography paper John Gordon was able to give an after dinner speech where, tongue firmly in cheek, he gave more details and insights into Alice and Bob. John Gordon's The Alice and Bob After Dinner Speech covers Alice and Bob, how mathematicians share jokes, how to represent the alphabet with words, and even why you should be nice to your calculator.

Math: What if pi is Wrong?

Now no one is suggesting that mathematics has been wrong for the last few thousand years. Don't worry pi still has a value of 3.14159265...  If you look at the formulas where pi is used though you'll find something interesting. More often than not you see 2pi (2 times pi). Maybe we should be using a value of 6.28318531... instead.

pi is Wrong! (pdf) by Bob Palais is where the idea hit the web first. Bob suggested a new symbol that looked like pi but has a 3rd leg in the middle to represent twice pi.

The problem is that a completely new symbol isn't available on any computer system and would take quite a while to become widely adopted.

Michael Hartl took up the cause of twice pi and decided to use an otherwise underutilized symbol. The letter tau. The Tau Manifesto is a lengthy explanation of why twice pi is much more useful than pi is. Filled with examples and derivations it is a useful starting point to incorporate the tau in your mathematics. Tau, or even just twice pi with any symbol, may never catch on. Sometimes the most rational and clear headed ideas get ignored and can't replace current conventions.

We'll just have to wait and see if tau starts gaining traction.

Math: You Can Do What With a Sphere?

Mathematics may be the most pure of the sciences. The most abstract of disciplines. And being the most abstract it allows for many problems that seem impossible or just plain flights of fancy.

One such problem is how to turn a sphere inside out.

Now we're talking about math and we're talking about the concept of a sphere. Don't think about trying to take a balloon and turning it inside out. It's not a real object mathematicians are talking about. A physical object can't be pushed through itself. A mathematical entity can. A real sphere can be torn and creased. A mathematical entity can't.

Even with a set of self imposed rules it turns out you can turn a sphere inside out (video).

Math: So You Want To Be A Mathematician?

With school starting this seems like a good time to find out what it takes to learn a good cross section of modern math. Which leads to the aptly names How to Become a Pure Mathematician (or Statistician) website.

The main page is the overview and contains a few notes at the bottom. I don't suggest clicking on the stage links (ie Stage 1, Stage 2, etc). Since the site is created using blogger the Stage links show the posts for that stage but not necessarily in the correct order. Clicking on the links below the Stage headings is much more useful. Elementary Stuff is where to begin.

The site itself is nothing more than a collection of links to books and resources. Direct links in the case of resources and books that are online and amazon links to the rest.

Even if you don't want to become a mathematician or a statistician give the site a look. The list of topics and required readings is overwhelming. No wonder it's hard to become a mathematician.

Math: Understanding the Fourier Transform

Play a pure tone on a computer and you hear a clean clear note. A perfect single sound. Take a number of pure sounds and you can create something quite complex.

Did you know that given a complicated sound you can decompose it back into the different tones (or frequencies) that make up the complicated sound?

The technique is called the Fourier Transform. It's not as complicated as it sounds. At least the overall theory is not too complicated to understand.

The DSP dimension has an article entitled The DFT "A Pied": Mastering The Fourier Transform in One Day. The on again, off again, on again blog called Physics For My Mom has a multi-part overview of the fourier transform.

• Fourier analysis 1 - Sound waves
• Fourier analysis 2 - More complicated sound waves
• Fourier analysis - Sines and integrals
• Fourier analysis - Sines and integrals (part 2)
• Fourier analysis - Sines and integrals (part 3)

That should be more than enough to get you started on fourier transforms. Once you understand that theory jump ahead and see if you can explain the math behind FFTs to me.