The FedEx Problem

But actually FedEx (Federal Express) was started in Little Rock. Fred had bought Little Rock Airmotive, a corporate aircraft interior decorating, customizing, repair, and lube and tune shop.

He dreamed of FedEx, and in a back room got a Dassault DA-20 Fanjet Falcon, popular executive jet, 28,660 pound max take off weight, two GE aft-fan turbo jet engines (originally designed for drones; used in the F-5; but with an aft-fan), a relatively rugged airplane, e.g., no frame fatigue life, worked on the modifications to make it into a suitable cargo plane, rewrote the official flight manual for the modifications, and got the FAA to approve the modifications and issue the appropriate certificate.

FedEx rushed to 33 such Fanjet Falcons; that was the initial plan for the full fleet, serving 90 US cities. The fleet schedule I did with my software and Roger Frock one night was for the 33 and 90. That schedule overcame some Board objections that were blocking some crucial equity funding, pleased the Board, "Solved the most important problem facing the start of Federal Express" (Fred, at a senior staff meeting), enabled the funding, and saved the company.

When I joined, the company was still in Little Rock.

Yes, the fact that Fred knew Memphis well was likely also relevant. But fully likely FedEx would have been in Little Rock if it had had a suitable airport.

There were some concerns about the Ohio valley: The whole place could get socked in with fog, quickly, nearly unpredictably.

For the hub location, I observed the decision making; the description I gave here is accurate enough; my criticism that the site selection was not just a geometry problem is fully correct; but I was not involved in the decision itself.

You mean when some angry union people showed up objecting to FedEx pilots handling packages?

You mean the time two barrels of liquids in the shop got confused and maybe some bad stuff got pumped by mistake into the hydraulic systems of some unknown number of airplanes?

The time I used the differential equation

y'(t) = k y(t) ( b - y(t) )

to please our two representatives of Board Member General Dynamics, have them unpack their bags and stay after all, and, thus, saved the company?

The time after midnight in my office I was practicing violin and, as I left, noticed that Fred, in the next office, had been working late?

The time at a top employee basketball game, in the locker room with Fred I asked him if he would do it again and got his answer, a more colorful version of investors don't like risk.

The time Fred, in the Dassault DA-20 Fanjet Falcon he saved as the company executive jet, was flying, kept finding airports closed, kept flying, and finally landed but flamed out from no fuel on the runway -- cutting it close.

How the planning had said that we could fly the airplanes half full and print money but actually flew the airplanes packed solid, doubled the rates, and still were losing money -- not good planning.

How Fred wanted to have containers to fit the inside of the plane, like regarding the inside of the cargo part of the plane as a big sausage and cutting it into a few identical slices. Then have ball rollers on the floor of the plane for moving the containers.

Then to move the containers on/off the plane, have a truck with a platform in front, about the right size for one container, able to be raised and lowered with hydraulics, drive up, next to the opening for the cargo door, and on/off the containers.

Well, Fred described all this on the phone to a guy in CA who got a truck chassis, did the welding and hydraulics, and drove the thing, chassis, no body, to Memphis.

Actually, at least as a prototype, it was okay. And the basic idea was important in production. Still, that the thing had been designed in a phone conversation upset some on the Board.

There were a lot of such war stories. But, in total the experience was a good lesson in doing a startup.

As others mentioned, such a great comment. Thanks for sharing graycat.

That kind of clarity and intuition is what drives so much technical decision making, because you rarely have the time and information you would want to make a decision.

I got involved for a while: I tried to do a continuous time, discrete state space Monte Carlo simulation, but it got too big to do at all easily and we dropped it.

So, the hub design was done with just back of the envelope arithmetic and engineering. It worked fine for years.

Over the years, the fleet scheduling problem went through various phases.

The first phase that worked at all well was just my software.

The last time I checked (I'd left for my Ph.D.), the package volume was high enough that one of Fred's ideas became the solution: At noon, have all the planes at distant cities. Then, in the evening, loaded up, all the planes fly to Memphis. Nearly all the flights are non-stop -- one plane, one flight, one city served. So, net, first cut, no scheduling problem at all.

Maybe there could still be problems for scheduling maintenance, considerations for remote sort hubs and/or international service, but that was all after I left.

Indeed, at one point I was starting to address the fleet scheduling problem with integer linear programming set covering, talking with G. Nemhauser, now at Georgia Tech, then at Cornell, etc. With current software and computers, what I was trying to do, for just the 33 Falcon airplanes and 90 US cities, should be nicely easy to do.

Another problem, though, could be the mix of aircraft types: Early on FAA and CAB rules boiled down to the Dassault DA-20 Fanjet Falcon with Fred's cargo modifications. Why? Because, e.g., wanting to be able to operate both airplanes and trucks, FedEx was flying as an unscheduled air-taxi, maybe intended for little Mom and Pop operations, but then could not fly planes with max gross takeoff weight (GTOW) over 25,000 pounds. Fred went to Congress and got his 28,660 GTOW approved as an exception.

Later Alfred Kahn deregulated airfreight and, thus, let FedEx fly nearly anything they wanted. At that point, there would be an issue of what mix of airplanes? But that was after I had gone for my Ph.D.

https://www.youtube.com/watch?v=YlVDGmjz7eM

Thanks graycat for taking the time to comment!

http://www.apress.com/at-work

Next time you're flying out of Denver, notice how long the takeoff roll is. When my father flew jets out of Denver, he'd hang the tailpipe over the back edge of the runway so he'd have every last foot of runway to get airborne.

But, yes, hot air at high altitude hurts jet engine thrust, a LOT.

Every couple of years, there'll be a day or two in summer when temperatures at the airport in Phoenix get above the maximum takeoff temperature for a lot of the smaller regional jets, and chaos ensues as all those flights (it's a US Airways hub) delay or cancel.

There's a leg connecting Tokyo and Mexico City. The Tokyo flight goes direct to Mexico City however the Mexico City flight stops in Hermasillo on the way to Tokyo.

Due to the higher altitude of Mexico City, it takes more fuel to get off the ground and requires a pit stop in hermasillo to get all the way to Tokyo.

Disclaimer I'm only partially sure that I'm remembering the whole story correctly.

When I was there, there wasn't much of a 'deadhead' problem, and I doubt that there ever has been:

Or it's roughly 8 PM in Seattle, SF, LA, Denver, Chicago, Cleveland, NYC, DC, Atlanta, Miami, etc., and the planes are all at such cities. Packages from those cities are loaded, and the planes fly to Memphis, getting there ballpark at midnight. The packages are sorted and the planes loaded. The planes fly back to the cities and get there ballpark 6 AM or so. So, so far no deadhead flights.

Well, might have a lot of lower priority packages didn't move from the origin cities, so load those on, fly to Memphis, sort near noon, load the sorted packages, and fly back to the remote cities. Still no deadhead flights.

At times, for some extra revenue, FedEx did some airmail flights for the USPS and/or some cargo charter flights, and, then, sure, there could be some deadhead flights, but such flights were so rare and unpredictable that getting human passengers would not have been promising.

UPS tried the same idea with 727s in the early 2000s but in both cases it proved too disruptive to normal operations.

However several European airlines today operate the inverse service with quick-changers, primarily passenger service and remove the seats at night for mail carriage.

Once virality started to take hold and the package volume grew, standing in the sorting center was an enthralling experience: The astounding, mind-bending variety of what the US economy wanted to ship with high priority was beyond comprehension and belief. Early on, time critical legal papers? Sure, there were plenty of those. Okay.

A cubic foot of some specific small metal fasteners a big production line needed ASAP or shut down at some cost of thousands of dollars an hour. Fine.

But bricks? Yup: A brick oven could send some samples to an architect, or an architect could send the samples to a customer.

Spare parts for computers, etc.? Yup. But baby chicks? Yup. Medical samples on the way to a medical testing lab? Yup. A live baby bear? It happened.

The variety was beyond all belief and comprehension.

No question about the need, definitely a must have -- a large fraction of the customers would have paid nearly anything, often beyond all reason.

> a complex multi-constraint optimization NP problem

There was a day when I thought that!

So, I was thinking non-linear optimization with, say, some of gradients, steepest descent, conjugate gradients, quasi-Newton, Kuhn-Tucker conditions, maybe some Lagrangian relaxation, etc. and saw no hope.

And I still remember the day when I got an explanation and learned better! Can make the problem 0-1 integer linear programming but otherwise linear. The non-linearities, really complicated costs, say, have a plane step climb when have a heavy cargo load, and really goofy constraints, e.g., don't fly over any US military bases or the White House, don't fly over residential areas at dinner time, etc., can handle in a simple way before the optimization itself.

The technique has some generality and, thus, is a good tool to have in a 'data science' tool kit, especially with current computer hardware, library software, and practical data availability.

Here's an outline of the technique: For one period from, say, 5 PM to midnight (again from 1 AM to, say, 8 AM) set up a table with one row for each city to be served, 90 cities, and one column for each candidate airplane trip, tour, from Memphis and back.

In this table, for each city and column put a 1 in the table for that city and column if the tour for that column serves that city and put a 0 otherwise. So, generate all such columns; may have some thousands of columns.

Considering all fine cost details want to account for, and taking expectations for random costs, say, from weather and air traffic control, find for each of the candidate tours its cost (first cut, assume that if a plane goes to a city, then it completely serves that city).

Then for the schedule, want to pick no more than 33 columns (there were 33 planes in the fleet) from the 1000s so that all 90 cities are served and total cost is minimized.

So, each column serves, covers, some cities (each city where the column has a 1), and want to pick columns that cover all 90 cities -- so have a set covering problem. Right, it's in NP-complete.

So, in particular, we have a linear programming problem: We have one variable for each column. The costs and those variables form the linear function of the variables to be minimized. The constraint matrix is just that table we described. The constraints are that each city is served by some one column.

Or in matrix algebra, we have from the table described above a matrix A with 90 rows and some thousands, say, m, columns. We have the variables x, m x 1. We have the m costs, 1 x m c. And we have the right hand side, 90 x 1 b where each component of b is 1. Then we want to find x to minimize cx subject to Ax = b. So, it's linear programming.

Then in addition we ask that the variables take on values of only 0 or 1 -- so we have a 0-1 integer linear programming problem.

If ignore the 0-1 constraint, then have a fairly routine linear programming problem but will likely end up with some fractional tours allocated.

For honoring the 0-1 constraints, on this problem likely could do well enough with branch and bound.

As FedEx grew, the nature of the scheduling problem changed. The outline above should have helped save significant bucks at least in the early years.

Then, of course, that solution technique could be used to evaluate the few candidate hub locations.

With some generalization, the solution technique could also be used to evaluate airplane types to add to the fleet.

So, here we have 0-1 integer linear programming set covering; keep it in mind and see if you can find another good application.

That's a multi-way tradeoff between local population and desirability as a destination (to ensure plenty of profit-making opportunities on direct flights); proximity to other important destinations (to offer short connections to non-hub cities); coastal or northern locations (good for trans-oceanic or long great-circle routes); climate (to ensure your hub doesn't get frequently shut down by adverse weather); congestion (to ensure your hub isn't frequently delayed by the sheer amount of traffic to coordinate); airport layout (to give you maximum efficiency for lots of flights); etc., etc.

When people wonder, for example, why Atlanta is consistently the busiest or second-busiest airport despite not being an obvious first-tier American city (those being New York, Chicago and Los Angeles), the weather, the congestion and the airport layout are huge factors. Atlanta doesn't get a ton of snow or severe thunderstorms like the northeast or upper midwest, it isn't in an overloaded air corridor like the northeast, and the airport's runway and taxiway configuration allow multiple simultaneous takeoffs and landings with much more ease than someplace like O'Hare (which is basically beyond capacity at this point with its configuration).

According to Wikipedia, seismic monitoring in the area wasn't installed until 1974, so at the time of FedEx's creation in 1971 it probably wasn't on their radar.

My offer (from Art Bass, then head of Flight Operations in part because he was, as the FAA required, a pilot) and offer letter said that (A) there would be a stock plan, (B) I would be part of the stock plan, (C) the plan would be based on salary in which case I would be quite high up, (D) the Board was considering the stock plan now and results were expected in two weeks, (E) if the plan were not equitable then the first plane out of Memphis would be full of ex-Federal Express employees.

With that, I joined, kept teaching my courses in computer science at Georgetown until the courses were over, at home got a time sharing terminal, a CP67/CMS account, etc., and dug into writing the software to schedule the fleet.

Some Board Members, including one with a lot of experience at American Airlines, doubted there could be a schedule. So, the Board wanted to see a schedule, say, for the full, planned fleet of 33 planes serving the full, planned list of 90 US cities. Some crucial funding, some equity, some loans on the planes, were being held up waiting for the schedule. The company was at risk.

I wrote the software, finished my teaching, drove to Memphis, and rented a room.

So, with the Board having doubts and the company at risk, one evening Roger Frock and I used my software to develop a schedule for the 33 planes and 90 cities. We printed out the schedule, had copies made, and handed them around.

Board Member General Dynamics had sent two representatives, one an aeronautical engineer and one a finance guy, to provide, say, adult supervision; those two guys went over the schedule fairly carefully and announced "It's a little tight in a few places but it's flyable" (pretty good from just a little fast work from Roger and I); CEO Fred Smith's reaction at the next senior staff meeting was "Amazing document. Solves the most important problem facing the start of Federal Express.". The funding was enabled. FedEx was saved. Pretty good from typing in 6000 lines of PL/I in six weeks while also teaching two courses!

PL/I is a nice language -- good on data types, data manipulations, data conversions, data structures, scope of names, exceptional condition handling, storage management, debugging, etc. E.g., its Based structures, can serve as a poor-man's classes in object oriented programming.

Later the Board wanted to see some revenue projections. I didn't want to get involved, but no one had more than wishes, hopes, dreams, intentions, etc. So, I started with the common high school question, what do we know? Well, we knew the present revenue or, if you will, number of packages per day. From our initial long term planning, we knew what revenue we expected with 33 full airplanes serving 90 US cities. So, in some at least a somewhat meaningful sense the desired projections were an interpolation between those two facts we did know.

Then the question was, how will the interpolation go? Well, why will the revenue grow? Sure: The revenue will grow as it has been so far, customers to be hearing about FedEx from current happy FedEx customers. E.g., maybe a customer to be gets a package via FedEx. So, we're talking word of mouth advertising or viral growth.

So, the rate of growth in revenue per day or packages per day should be directly proportional to (A) the number of current customers and (B) the number of customers to be. That is, the rate of growth should be proportional to both (A) and (B), that is, to their product.

So, all downhill from there: At time t, let y(t) be the revenue, say, per day, at time t. Let t = 0 correspond to the present. So, we know y(0). Let b be the revenue per day with a full system, that is, 33 full airplanes and 90 US cities. That is, we know both y(t) and b.

So, from freshman calculus, the rate of growth is the first derivative of y(t), that is, d/dt y(t) = y'(t). So, from the proportionality, we have that there must exist some constant k so that

y'(t) = k y(t) ( b - y(t) )

So, we have an initial value problem (we know y(0)) for a first order, linear ordinary differential equation.

Okay, how to get a solution? Easy, just need freshman calculus, not even a course in differential equations. And, yes, there is a closed form solution, right, with some exponentials.

Right, the solution is the famous logistics curve sometimes seen as doing well tracking, say, the growth of TV set ownership in the early years of TV. So, my derivation, as just above, can be seen as an axiomatic derivation (maybe rediscovery, maybe original) of the logistic curve. The solution may remain an okay, first-cut approach to understanding viral growth.

So, I showed my work to Senior Vice President Mike Basch, likely the one most responsible for getting the projections for the Board, and he liked my work. So, on a Friday afternoon we picked several candidate values for the constant k and drew the corresponding graphs of the revenue projections. We used my HP calculator, reverse Polish notation, stack machine, etc. -- HP should run an ad! We picked a value of k that gave what seemed to be reasonable projections and declared the problem solved.

The HP? It's still in my center desk drawer. Checking, right, it's an HP-35. My wife and I paid $400 for it.

The next day, a Saturday, at about noon, I was in my office likely working on fleet scheduling and got a call from Roger Frock asking if I knew anything about the revenue projections. Saying I did, he asked if I could come over to the HQ and explain.

So, I got into my Camaro hot rod (396 big block, etc.), and drove over. Yes, I brought my HP-35.

As I arrived, at one of the old WWII wooden hanger buildings, people were standing around and not happy. Our two guys from General Dynamics were standing in the hall with their bags packed and not happy.

Roger led me to a table with the graph, picked a point in time, and asked me to calculate the value on the graph. So, with my HP-35, I punched the buttons, stopped, slowed down, cleared the HP-35, started again, slowly and carefully punched the buttons again, and got the value on the graph. I did that for several points for the graph, and then everyone started to get happy.

It turned out that the Board meeting had been that morning; Mike Basch was traveling; I'd not been invited to the Board meeting; the graph had been presented; the two guys from General Dynamics (GD) had asked how the graph had been calculated; and everyone else at the meeting dug in trying to answer. They worked for hours with no results. Finally the two guys from GD lost patience with FedEx, returned to their rented rooms, packed their bags, got plane reservations back to Texas, and as a last chance returned, with their packed bags, to the FedEx HQ to see if anyone could explain the projections.

Somehow Roger Frock had guessed that I'd done the projections, called me, and got me to the Board meeting just in time.

It was close, but I'd saved FedEx a second time.

Right: Some people in FedEx would rather have seen FedEx go under than invite me to the Board meeting. We're talking some severe cases of jealousy, bureaucratic infighting, attacking the guy down the hall instead of the competition outside of the building, goal subordination as in organizational behavior, etc., right? Bummer.

Right: Apparently I was the only person at FedEx who still understood freshman calculus. Gads. And I never even took freshman calculus, taught it to myself from a book, and started with sophomore calculus.

I never got any thanks for saving the company the second time.

I'd been at FedEx for over a year. I had been commuting every few weeks home to Maryland for a few days at a time -- not good. There had been no more about the stock that had been supposed to come in "two weeks". The company had some problems, e.g., had nearly gone out of business due to not inviting me to the Board meeting. Also the basic planning was way off -- the planning said that we could fly the planes around half full and nearly print money, but we were flying the planes packed solid, had doubled the rates, and still were losing money -- bummer.

So that I could be a good bread winner in my marriage and for our kids if we could have some, as we hoped, I wanted something valuable no one could take away from me, a Ph.D. for my career and/or stock.

So, I'd gotten accepted for an appropriate Ph.D. at Brown (Division of Applied Mathematics), Cornell, Princeton, and Johns Hopkins.

The oil crisis hit. Saving money, especially on jet fuel, became a biggie. So, I was working on that. I was getting a lot of flack from others, especially my manager.

Finally I called a meeting to explain what I was working on, three projects. My manager said I couldn't do that because he was busy and couldn't come. I told him, fine, then don't come.

He came. So did Fred, Roger Frock, Art Bass, the top 15 or so people in FedEx. My manager was sitting next to Fred and kept objecting to what I was saying until Fred told him to cool it.

One of my problems was to use deterministic optimal control theory to say how to climb, cruise, and descend the planes.

A second problem was to use 0-1 integer linear programming set covering to develop schedules that would save on OpEx and maybe also CapEx.

A third problem was how to buy fuel during a trip from Memphis and back. So, broadly the idea was to buy extra fuel where it was cheap and carry it to the next stop or two where fuel was more expensive. We were getting fuel for $0.16 a gallon in Memphis but paying up to $0.55 cents a gallon (in Nashville). So, that's a case of what has long been known as fuel tankering. But doing that interacts with how to climb, cruise, and descend the airplane, not being late in the schedule, loads, weather, air traffic control, etc. And typically a lot of the cheap fuel gets burned off just from trying to carry it, and how much gets burned off has a lot to do with the flight plan. And any such decision to buy extra fuel is a bet on the future of the trip back to Memphis, that is, a bet against the random package loads, weather, air traffic, etc.

So, how the heck to solve that? And, for various reasons, couldn't get a solution from carrying a computer on the plane and, really, not even from using a computer on the ground after landing. I'd found a way!

So, Fred put me under Senior VP of Planning Mike Basch and, thus, made me Director of Operations Research.

But the fall came, and I had to decide actually to leave for graduate school or not. With no stock, not a lot of thanks, with a lot of scars from being attacked, still away from my wife, the company still at risk, I decided to go to graduate school. I liked FedEx, the challenges, the work, etc., but making the stockholders rich, with me not one of the stockholders, while wrecking my marriage and passing up the chance for a Ph.D. that might help my career and that no one could take away from me looked not good. If I couldn't get stock with the company still at risk and worked to make the company valuable, then what hope would I have of getting stock in the company I'd helped make valuable before getting any stock?

I went home to Maryland. At the last moment, Fred wanted me back in Memphis. He and I met with Mike Basch, and Fred said, "You know, if you stay, then you are in line for $500,000 in Federal Express stock?". Heck no; I didn't "know" any such thing; I had had and accepted such promises before, "two weeks", and after 18 months, saving the company twice, and with three projects to do much more for the company, all there was were more such promises, not on paper that a lawyer could do something with, no thanks -- "Fool me once, shame on you. Fool me twice, shame on me.".

Sure, that $500,000 would be ballpark $50 million to $500 million today. And apparently some people did get some stock. But there that last day, Fred still was just not putting it down on paper.

Since then I ran all this past a lawyer who concluded, "Legally FedEx owes you nothing. Morally they owe you everything.".

So, here on HN, maybe I definitely should tell this story as I have so that others can benefit so that more promises of stock can become ownership of stock.

Of course, there is a lot more to getting wealthy from stock in a startup than what I've outlined here.

Broad Lesson: The broad lesson for people in startups with promises of stock, become very well informed and be very careful.

My reaction: Do my own startup. Doing it. Need to get back to it. It'd be fun to make more money than Fred! I have a shot! Back to it!

As elsewhere on this thread, I'm doing a startup.

What I've posted to HN in this thread and otherwise is about the best I can do for now.

For the history of FedEx, there are several books already. Roger Frock wrote one. I've seen none of them, but some friends found me mentioned in one of them. I was also mentioned, with some errors, in an article in Informs, the popular magazine on operations research and management science.

No time to retire yet!

If I get rich, then I will have several projects to consider. Sure, several projects of the kind *what I wish I'd known when I was 8, 10, 12, 14, 18, 22, 30, 40" are candidates.

(Drats, it was a nice one.)