Greater than 5 minutes, my friend!

About 2 years ago I supported a strange translation project. I’m always in for a challenge. My customer had a lot of plain text data from mixed sources — that was intentional and by design. I had to advice on how to outsource the work so that quality control was not going to be a huge cost. They could live with 10% of the translation cost.

The first problem I faced was that the translation budget was actually not very high. Well, … that was my “expert guess”. Some of the translations were pre-translated by reliable but maybe outdated memories, another part came from less reliable online TMs, and a last, small part was from a generic MT engine: the perfect mix for disaster. But being judgmental would not help to convince the customer that he was willing to buy too cheap, exploiting translators. So, I send out a couple of small jobs to translators I knew well and asked them to work on the job at their own conditions, but allowing me to register how much time they spend on each sentence of the job. Turned out that somehow the pre-translation was much better than I expected. Till today I don’t know why that was the case, but sending out the test jobs to people I trusted (and translators that trusted me), helped me to understand that the low word rate was OK. The translators accepting this work would make a decent living and I should not feel bad about being part of this project.

The customer was paying between 5 and 7c/word and for EVERY word, even the high fuzzies and 100% matches were paid this way. The reason was simple: since a solid part of the job was generated from unknown sources, no reason justified to buy any other way.

The REAL problem however is that 10% of this rate, would lead to almost no budget for QC.

10% of budget = QC 20% of translation

I tried to convince the customer that we had enough budget for checking maybe 20% of the job. I failed. He really insisted on having everything checked. Even pre-processing what should be checked by humans (using a QA tool), could not convince him.

I called some translators asking for their advice. Nobody had an idea how to handle this.

If I would send the job only to the translators that I worked with for many years, checking 20 % of all their translations, we would need almost year to get all translated and checked. Not an option. We simply had to send a large part of the the job to translators that we did not know.

So basically we now had 2 problems: too much work for all translators I could trust, and not enough budget to organize QC the way my customer wanted.

Revealing the need for QC

So the only thing I could do was to show my customer WHY he had to budget for doing 100% of QC OR accept to do partial QC. I created a small translation tool that was basically reading and writing to an encrypted SQLite file. My application was also registering how much time people were spending on each segment, how many times they reviewed a segment and their IP address. This was also mentioned in the readme file.

What embedding this metadata revealed:

• 50% of the translators did not even open the readme (only half of an A4!). This was shocking.
• Our jobs started to travel a lot: we outsourced to translators in the country where the target language was the mother tongue of the translator (I could have written “native speakers”, but native speakers could live everywhere: I outsourced to French translators in France, German translators in Germany, Italians in Italy…); Yet my jobs started to travel. The people spending most time on the job, did the work themselves. But about 30% of all jobs was sent to people who were super-efficient: their production speed was up to 10 times as high. Woow… I also noticed they lived in India, China, Malaysia, Brazil…
• When doing a post-calculation on the translations done by people living in the countries we outsourced to, I saw that the word price was correct. The production speed made up for the low word rate and the translators confirmed they were really happy with the jobs.
• When doing a post-calculation on the jobs that had travelled (and that were sub-contracted and even sub-sub contracted), I noticed that the distance created was almost random. Digging in, I found unmodified Google and Microsoft translations, but also other translations that I could not trace back to a source. But when a huge distance is created in a couple of seconds… I have reasons to doubt the quality.

I presented the customer the results of this experiment. It became clear to them that 10% was not enough budget for checking 100% of the translations. But… as my experiment had shown: with some metadata I could create a metadata filter showing quality issues even without spending anything on QC.

[BETWEEN BRACKETS: as the word price turned out to be really OK after all, we could have opted to reduce the word price for translation in order to create some extra budget for QC. I know this is something some translation companies sometimes do. We did not want to do this as we felt it was more valuable that our translators were paid correctly: lowering the translation price, would simply increase the risk that translators would not spend enough time on the job, outsource more, check stuff less themselves… QC should never eat the translation budget.]

What happened next

The customer decided 10% was good enough for creating process control, and for spot checking the work of the translators who were spending the expected amount of time on the job. This spot checking revealed 2 things:

• translators who deliver good quality usually deliver good quality.
• the other translators were easy to filter out.

The actual QC on the translations itself became pretty sophisticated (I cannot reveal any details about this) and 10% of the translation budget did give us certainty on a large part of the translations. The customer decided the remaining part was not worth an extra investment. The quality risk was acceptable because we were transparent about our process. Being open about our process, aligned our project setup to our customers’ risk appetite.

The reliable translators who worked on this project, are still my favorite translators. I recommend them to all my customers.

Lessons learned

1. QC is a process first, a job second.
2. Nothing travels as light as a translation job.
3. QC is not optional. It is a bare necessity.
4. Process transparency may buy us the support of our freelance translators and our customer.

 

 

Featured Image: Augustus Egg, The Travelling Companions (1862; Birmingham Museum and Art Gallery).