My Neural Machine Translation Project – Prologue

Lately, when I introduce myself as a translator, or more specifically, as a patent translator, people invariably always ask me whether I’m worried that I’d be replaced by neural machine translation (NMT) in the next few years. Obviously, this being Silicon Valley with its ubiquitous self-driving cars, drones, and robot security guards, I can’t just reply no, point to the latest MT translation error meme that is making the rounds on social media and be done with it. Here, a deeper (pun intended) argument is needed. In addition, the European Patent Office (EPO) announced their new Unitary Patent, which is supposed to reduce translation costs for applicants significantly, whereby many now mandatory translations of patents are to be replaced by machine translations. This new Unitary Patent was supposed to go into effect on January 2018; however, it currently looks as if this timeframe will be delayed.

Nevertheless, my inbox is also beginning to fill up more and more with offers for post-editing of machine translation output (MTPE). I am not the most efficient editor when I am editing translations by human colleagues, even when the text is excellent, because I tend to get sidetracked by matters of style. Thus I am utterly unsuited for MTPE, because I simply lack the patience to deal with nonsensical machine errors. However, in light of all of the above, a plan began to form in my head: I want to set up my own machine translation engine.

While this is certainly very ambitious, it’s not impossible. I have years/decades of background in advanced mathematics (theoretical physics) and computer programming. Furthermore, there are now several open source NMT toolkits on the market, complete with various libraries and discussion forums. Obviously, I could just download one of the toolkits, train the net with various open source corpora and be done with it. But that would be too easy! And not very productive. I want to get to a point where the net is trained well enough so that I can actually use the output in my daily work. I also want to achieve an expert level where I understand how NMT actually works, perhaps to work as an NMT consultant instead of an MTPE slave when the NMT apocalypse descends on the translation world (which is not likely to happen anytime soon). In addition, I will document my experiences on this blog. Since this is a side-project, I can’t promise to blog regularly, because my progress will be highly dependent on my daily workload. I certainly won’t be “live-blogging” due to the inevitable R-rated Austrian expletives that will accompany the programming stage.

I began the journey over a year ago by taking an introductory class by Andrew Ng on Machine Learning on Coursera. Andrew Ng is not only the co-founder of Coursera and a Stanford professor, he is also an excellent teacher. The course introduced all the necessary concepts with just the right amount of math (for me as a physicist) and programming (in the symbolic language MATLAB). I highly recommend this course as an advanced introduction for anybody who is interested in the topic. However, note, that Andrew Ng’s excellent course does not cover machine translation. I followed this up with several courses on Robotics (on Coursera) and on Artificial Intelligence (on EdX) at the introductory Master’s level. I even built an autonomously navigating robot, nicknamed Boticelli. While I am far from an expert now, I certainly know more than the average amateur about artificial intelligence and neural nets. I will summarize what I’ve learned so far in a presentation at the 58th Annual Conference of the American Translators Association this fall.

The next steps will be to buy the necessary computer hardware and pick one open source NMT toolkit. Neural nets require dedicated hardware, that is, very high-end graphics processing units (GPUs), because the training phase of neural nets basically consists of huge numbers of matrix multiplications. Dedicated GPUs are capable of performing large numbers of computations in parallel, in contrast to CPUs, which are best used for serial computations. Thus, to set up an NMT engine, a “gaming” PC with a VR-ready high-end graphics card is necessary, because ironically, the computations for virtual reality computer games and the computations for neural nets in serious applications such as translation are quite similar.

But more on that in the next post. Stay tuned!

Translation Blues

Inspired by Janis Joplin’s “Mercedes Benz.”

Oh Lord, won’t you give me a text that makes sense?
The deadline is looming, I must make amends.
Worked hard all the week long, no help from my friends,
So Lord, won’t you give me a text that makes sense?

Oh Lord, won’t you get them to pay me the fee?
Dying for Dollars, ain’t working for free.
I wait for delivery each day until three,
So Lord, won’t you get them to pay me the fee?

Oh Lord, won’t you spare me another meltdown?
I’m counting on you Lord, please don’t let me down.
Prove that you love me, I’m worked to the ground,
Oh Lord, won’t you spare me another meltdown?

Oh Lord, won’t you give me a text that makes sense?
The deadline is looming, I must make amends.
Worked hard all the week long, no help from my friends,
So Lord, won’t you give me a text that makes sense?

Artificial Intelligence, Machine Learning, and Neural Networks

This is the title of my presentation proposal for ATA58, the yearly conference of the American Translators Association (ATA), which will take place in Washington DC from October 25th to October 28th. The proposal was accepted and my presentation is preliminarily scheduled for Friday afternoon, October 27th, in the science and technology track.

artificial brain

I became interested in the topic after hearing various predictions that the translation profession will be completely taken over by machine translation (MT) by around 2020. Since I found these doomsday scenarios a bit exaggerated, I wanted to learn more about the topic. After taking several online courses on Coursera and edX on the subject, including several at the Master’s level which involved quite a bit of programming, I am now able to form my own opinion. I am confident that human translators, or at least certain segments will be needed in the near and intermediate future beyond 2020, despite the enormous advances in neural networks and neural machine translation. And by the time neural nets reach the so-called “singularity,” that is, they are able to think like humans, we will all be obsolete unemployed.

Artificial intelligence is a fascinating topic and I am looking forward to the opportunity to talk about it in the science and technology track at ATA58. However, I have not yet taken a course on natural language processing (NLP), which is the science behind MT. Thus I will not discuss NLP or MT, but I am considering this as a possible topic for a presentation at ATA59.

Below is the abstract of my upcoming presentation:

Artificial Intelligence, Machine Learning, and Neural Networks – an Introduction

From spam filters to stock trading bots, the applications of artificial intelligence are already omnipresent. This poses important questions such as: Will my autonomous vacuum cleaner go on a rampage and eat the hamster? Do neural networks think like brains? What are the chances of a robot uprising? The presentation will address these questions and give an introduction to artificial intelligence, which is impacting all our lives, perhaps more than most people are aware of. However, the talk will not discuss machine translation and related topics. No knowledge of computer science or advanced mathematics is required to attend.

Scam Alert! Corporate Compliance Services

Today, another scam reached my inbox, my postal mail box, to be precise. A letter by “Corporate Compliance Services” sent me a “Labor Law Compliance Request Form,” notifying me that I am required to pay $84.00 to comply with federal law 92 USC Sec. 999 etc. etc. and so on and so forth.

Corporate Compliance Services scam

Corporate Compliance Services scam

Obviously, they messed with the wrong translator again, because in this case it was nearly immediately clear that this was a scam, as an internet search instantly revealed.

Clue number 1: The scammers sent the notice to my home address, not my business address. I use my business address for every single correspondence with all government entities and agencies, except my personal tax return. All business-related correspondence with city, county, and federal agencies as well as my clients goes through my business address.

Clue number 2: While the letter looks really elaborate, in the front and in the back, even more official than some correspondence I received e.g. from the IRS, they misspelled the word “address.” No goverment agency misspells a word on an official form. Some individuals might send out misspelled posts on social media, but no form I have ever seen contained such a blatant spelling error.

I was tempted to reply to the fake “notice” with a letter instructing them to deduct the fee from my Nigerian bank account, but alas, several deadlines are looming. Plus that would confirm my personal address.

Fraudulent website update – happy ending

A bit over a month ago, I discovered thanks to Google Alerts, that a fraudulent website was using my business name and excerpts of my copyrighted website content without my permission to advertise their dishonest services. The site has been taken offline a few weeks ago and has been offline ever since.

Site offline

Fraudulent site offline

Here is the timeline of my actions which led to this positive outcome, which may help you if you are ever in a similar situation.

Step 0: Set up Google Alerts

If I hadn’t set up several Google Alerts to inform me whenever my name or my business name appears on a new site online, I would have never known about the impostors. I wrote a blog post about how to do that here.

Step 1: Post a disclaimer on my website

I posted an alert immediately after the discovery of the fraudulent website, in which I disassociated myself and my business from the website and all its activities. If I had had the slightest suspicion, that the impostors could contact my existing clients or solicit new clients under my name outside of that website, I would have also proactively contacted my existing clients and posted another alert/disclaimer on all my public and semi-private social media and professional accounts.

Step 2: Find out who is behind the website

Unfortunately, this step proved to be quite difficult, because the real host of the website was hidden under several layers of anonymized entities. I began by looking at the internet registry information, which you can find via any Whois domain service, for example The domain and the network whois record indicated that this particular website was registered in Panama. Unfortunately, contacting the registrar (see step 3) proved not very useful, because they claimed they were only responsible for registering the domain name, not for the content. I was referred to another entity in China, which also claimed not to be responsible for the content.

However, the domain name registrar was helpful enough to suggest to run a ping traceroute, which gave me the domain name and the IP address of the entity that actually hosted the content on their servers. One such service is for example The last entry of the route is the IP address and domain name of the server that I was looking for. With this information, I went back into the Whois domain lookup and got the record of the actual host. The host is supposedly based in Canada, but the IP address of the server is actually located in Utah, USA, as an IP location service such as revealed. Now I had enough legal ammunition to take action.

Step 3: Cease and desist letter

After peeling back all the layers of the onion, I sent a very official sounding cease and desist email to the aforementioned hosting service. Actually, I had sent cease and desist letters to all the involved parties/layers, but although I received nearly immediate responses, they just referred me to the next layer. But when I sent a cease and desist letter to the hosting service, the site got taken down the very next day, although I never received a response from the hosting service!

Regarding the content of the cease and desist letter, I looked up a template online for the proper legal phrases. I mentioned that my business name, which was used fraudulently, is registered in the State of California (since 2010), and that the content of my website, which was used without my permission, is copyrighted. More on that below. I was prepared to take further action, but luckily, this was not necessary.

Step 4 (not taken): Invoke US copyright law

According to US Copyright law, all work is under copyright the moment it is created and fixed in a tangible form, without actually having to register for a copyright. Aside from the standard “All rights reserved” disclaimer on my website, I also use a WordPress plugin to take snapshots of the content that I deem worthy of copyright. This serves as “fixing it in a tangible form,” as required per the US copyright law quoted above. There are many such plugins available.

If the site had not been taken down, I was prepared to send a takedown notice according to the US Digital Millennium Copyright Act to the service provider that was hosting the offending site. The steps to send a DCMA takedown notice are described here. Luckily, this was not necessary.

All is well that ends well. I want to thank my colleagues who helped me peel back the layers of this fraudulent onion. I hope that I can help other people in a similar situation by sharing this experience. I also hope that potential criminals will be sufficiently deterred to try something similar in the future.