When I worked on Google Offers (a Groupon clone), we had a user
interaction problem. When a user purchased an item, there were
three important steps that we wanted to acknowledge:
- When Google recieves an order. The user should get positive
feedback that the order was recognized. If the user doesn’t get
this, they will either re-submit the order, or be surprised when
the order is fulfilled. The order is placed on a queue.
- When Google processes an order from the queue. The user should
get positive feedback that Google is working on fulfilling the
order. Usually, an order is fulfilled reasonably quickly, but there
are situations where critical systems are unavailable and the order
has to remain in the queue for an extended period of time (several
hours). The user gets nervous if they get ghosted by the system
after getting positive feedback that Google got the order.
- Finally, when Google has made the decision whether to fulfill
the order or has declined the order. The user needs to be told
whether to expect shipment or a refund. If a refund, then the user
can take action to re-submit the order.
So in submitting an order to Google Offers, a total of three emails
would be sent to the user so they could watch their order proceed
through the system. The sending of these emails was controlled by
the “commerce” API. The commerce API was
a walled off section of the Google infrastructure that knew how to
talk to the credit card companies and charge money. Normal Google
code was not allowed to do these things but had to work via the
commerce API, and the commerce API would take care of ensuring that
the appropriate pop-ups would appear on the user’s screen and that
the credit card information was securely obtained. Normal Google
code never got its hands on the user’s credit card information, it
only got a "go/no-go" signal from the commerce API.
So the commerce API would be the system actually taking the steps
of recieving the order, charging the credit card, and returning the
go/no-go signal to our system. We would instruct it to send email
for each of these steps. So far so good.
The problem was that often the order would go through very quickly.
The emails were processed in batches, so the email that acknowledged
the reciept of the order could end up being received after the email
that acknowledged that the order had been fulfilled. The user would
first get an email saying "We charged your card." and only after this
would they get an email saying "We got your order." This would
confuse the user.
There was no way to add an artifical time lag, nor would we want
to. We could not guarantee that the emails would arrive in the
right order. (Even if we could, we couldn’t guarantee that the user
would read them in the right order.) The solution that I proposed
was to explicitly number the emails so that each email would say
"This is email number N of 3 expected emails." and even perhaps a
small message that said "Emails can arrive in the wrong order." If
a user got email 2 first, then email 1, they could figure it
out.
But the product managers didn’t see it that way. As far as they
were concerned, it was confusing when email 1 arrived after email 2,
so they wanted to not send email 1 if email 2 had been recieved. This
is a nice idea, but I pointed out that we had no control over the
order of arrival of emails, so there was no way to know if email 2
would be received prior to email 1 at the time we were sending email
1. They were adamant: "Don’t send the second email (that is, email
1, which arrived second in some situations)."
Ok, then. I adjusted the code to suppress the sending of email 1.
This solved the problem of email 1 arriving after email 2, sure, but
recall that email 1 was the "Google has received your order and has
placed it on a queue to be processed" acknowledgement. Now when
people placed an order, they would no longer get confirmation that Google
had received it. Usually, Google would process the order in a
timely manner and they’d quickly get email 2 which said "We are processing
your order", but if there were some reason that we could not
process the queue for some time, the user would be left in the dark
about whether Google had heard them in the first place.
Complaints poured in.
The subsequent conversation was something like this:
“Why aren’t you acknowledging that the order has been
received?”
“You explicitly told us to not send that email. See this
communication (cite reference).”
“But that was not what we meant. We meant, don’t send it if
the user has received the ‘Your order has been
processed.’ email.”
“How are we supposed to know if email system
delivered his mail and that he read it in the right order?
We’re good, but not that good.”
“You mean emails can arrive or be read in the wrong
order?”
“Exactly what we said in this communication (cite
reference).”
“...”
“May I suggest we number the emails so that the user can
figure it out if they arrive in the wrong order?”
“No, don’t do that. Just put it back the way it
was.”
Done, and we are back to the original problem.
Out-of-order packets are an old problem that existed and was
solved before computers. Computer programmers are, of course, very
aware of the problem and the solutions. Computer programmers are
well versed in the problems of “process”, and when
laymen try to play at being programmers by interfering with process,
they generally screw it up.