I work at Fourthline, a company that manages the Know Your Customer (KYC) process for companies like Revolut, TradeRepublic, and N26.
For those who don’t know what KYC is, think of when you register online to use one of these services and are asked to take a selfie and a picture of your ID document to prove you are who you claim to be.
That’s KYC in a nutshell, and as you may guess, we don’t want fraudsters to slip through. If you’ve ever played Papers, Please, you can imagine this isn’t always easy.

Fraudsters are getting better, and technology is helping them.
Deepfakes and AI-generated selfies and documents have become some of their tools, and my job at Fourthline is also to fight this kind of AI.
Yes, I’m an AI engineer who fights AI. A sort of cyberwar destined to become even more challenging as technology advances.

But to fight deepfakes, I should also study how similar problems were handled in the past.
Image manipulation has existed for decades, long before generative models reached today’s quality.
So let’s go back to 2007 and talk about compression and Error Level Analysis (ELA).

Compression

Almost every image on the internet is compressed; handling uncompressed images everywhere would be too expensive.
Let’s look at an example.

I took an image with my camera with a resolution of 6000x4000.
Quick math: that’s 24,000,000 pixels. Every pixel stores information about color, and color in computer science is usually expressed with 8 bits.
To cover most colors, we need three channels — red, green, and blue — so every pixel carries 8 × 3 bits of information.

The weight of this image should be 6000 × 4000 × 8 × 3 = 576,000,000 bits, or 72 megabytes.
Instead, it weighs only 6 megabytes.
Why? JPEG compression.

JPEG compression and artifacts

JPEG is the most widely used image compression algorithm.

In simple terms, the algorithm works by removing parts of the image that are barely noticeable to the human eye. It has a parameter called quality that sets how aggressive the compression is.
A quality of 100% means very little is removed, while 1% means heavy information loss.

Because information is lost, the image content is altered in subtle ways. If you look at the single pixel values, you’ll notice differences. Depending on the chosen quality, these differences may be invisible to the eye but still exist.
These differences are a trace left by the JPEG compression, a sort of fingerprint. We call them artifacts.

Now the interesting part comes.
Since JPEG compression is applied to the entire image, these artifacts should always be consistent inside the image. If the image is tampered with in some way - say an object is removed in Photoshop - this will affect the JPEG artifacts as well. You would notice an inconsistency of these artifacts in the particular area of the image that was altered.
And this is exactly what ELA is based on.

Error Level Analysis

In 2007, Dr. Neal Krawetz introduced ELA (among other techniques) in his talk A Picture’s Worth….
The idea is simple:

• Take the image to analyze
• Compress it again with JPEG at a chosen quality - this will become a parameter of ELA
• Calculate the difference between the two images

The result is an ELA map to interpret.
The assumption here is that the image (or parts of it) was compressed with JPEG at some point.

Let’s use a toy example to explain how it works.

Toy example

Using something from my domain of work, let’s consider this specimen of a French national ID card.

Suppose I know it was compressed at JPEG quality 95%. I’ll use that later.
Now, I inpaint the image with an AI model and get this:

And, importantly, I save this deepfake in PNG format.
Unlike JPEG, PNG uses lossless compression, meaning no information is lost.

Now, using the information above, we perform ELA with a JPEG quality of 95%. In this way, the part of the image that was already compressed with the same quality factor should not produce any new artifact.

The tampered area is obvious. We successfully detected the deepfake ID and blocked a fraudster.

But in reality, you may not know the original JPEG quality, and it’s unlikely that the manipulated image (previously a JPEG) would be saved as PNG.

Real example

Now let’s assume the deepfake was saved as JPEG.
We don’t know the quality factor. We only know that if the image was altered, part of it will be double-compressed and part single-compressed, producing inconsistency.

We perform ELA with JPEG quality 80% for both images.

Now ELA feels more like art.
There’s a clear difference, but without knowing tampering occurred, could you detect it from the map alone?
To better understand how to interpret an ELA map, we can take some notes from how JPEG works:

• JPEG creates more artifacts in high-contrast areas (like text or borders), making ELA brighter there.
• Flat surfaces should have uniform ELA values.
• Similar textures should produce similar ELA.

If some part of the image doesn’t respect these ‘rules’, then it might be suspicious. Like the deepfaked face on the ID, with a suspiciously bright ELA.
For more tips, see FotoForensics and their tutorials.

Conclusions

So, can ELA be a solution to detect AI-manipulated images?
No, not by itself anyway.

ELA is just a tool in the Image Forensics toolbox that can help fight deepfakes.
But many other things can be used together with ELA, such as EXIF information, PRNU, PCA and so on.
And probably some of these techniques will be the topic for another post.

“AI agents are getting so smart nowadays, my job will eventually be replaced by them. You don’t have to worry about this instead. You are an AI engineer, you create AIs, you cannot be replaced.”

My face takes a specific expression at this point, showing some sort of discomfort, mixed with a “how do I explain this now in simple terms?” kind of face. The kind of face you have when you meet a stranger in the elevator and you don’t know how to act.

The fact is, it will be very likely that my job will be replaced sooner than most developer jobs.

Let’s dive into the reasons why I think this, starting by defining what an AI engineer is.

What exactly is an AI engineer?

The idea for this post came after seeing this image on my LinkedIn feed:

This excerpt was posted by Arvind Narayanan, and it comes from his book “AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference”, written together with Sayash Kapoor.¹

Although I didn’t read the book myself, I think this introduction explains the current situation with AI — and AI engineers — very well. At this moment in the world, artificial intelligence is everything and nothing at the same time.
ChatGPT is AI, the software that post-processes images on your camera phone is AI, the algorithm that selects the best commercials to show you on social media is AI, the non-playing characters on your latest videogame are AI.
And yet, these four types of artificial intelligence are very different from each other.

ChatGPT is an LLM, a transformer-based architecture with billions of parameters.
The image processing algorithms on your phone are likely small and efficient convolutional neural networks designed to work with the computational power of your (relatively) tiny device.
Recommender systems are quite complicated, and they could use many different kinds of technologies specifically engineered for the field of application.
The AI of NPCs in video games are often just classical pathfinding and search algorithms, like A*.

Technologically speaking, the knowledge you need to develop any of these AIs is different from each other. Sure, some of the underground fundamentals are the same — in the end, neural networks are almost everywhere — but going back to the figurative examples used in the image above, you can say that both the car and the bike use wheels, but you wouldn’t call a bike shop to fix your car’s wheel — or a car mechanic to fix the engine of a rocket.

So how do we differentiate between these AIs? The answer is: we don’t.
Currently, everything is marketed as “Artificial Intelligence” without any clarification of what this means. And this affects the job title too; a person having the “AI engineer” title could literally do everything. I have even seen many people claiming they were AI engineers while they were just using ChatGPT APIs to build their software.

If I start looking for a job as an AI engineer, I would receive job offers of any possible kind of AI out there that I don’t even care about.
On the opposite side, since I mainly work with images, I could say I’m a Computer Vision Engineer, and I would receive tons of job offers that have nothing to do with AI, since Computer Vision is a far wider field than only AI.

So, what exactly is an AI engineer? Well, I came to the conclusion that it is whatever you want it to be.

Why are you worried that you will be replaced?

It’s an understandable question. I just said that AI is an umbrella term that encapsulates many different kinds of subjects. LLMs like ChatGPT are only a part of what is called AI.
So why should I be worried?

The thing is, these LLMs and foundation models are becoming so general that they are starting to include way more domains under their “side” of the AI world. Just a few days ago, Meta released their new version of DINO, a versatile, powerful and efficient vision model that could be easily applied to different tasks with very little work. Without using annotations and with very good performances. No more need to do research about a certain topic — you already have a plug-and-play solution that will probably work for many applications.

This is the direction AI is taking: cannibalizing all the other branches of AI into general models that can be applied to everything. Why bother tailoring a solution to a specific problem when you can just use the latest generalization breakthrough from big tech? We’ll eventually reach a point where having AI engineers and researchers will no longer be convenient for most companies. The best AI researchers will be concentrated in big tech, and the rest of the market will be highly saturated. Tailored AI solutions will become a luxury that most companies will happily avoid.²

Luckily, this isn’t the case yet. General models still aren’t specialized enough for many domains — especially when data from those domains is scarce or difficult to obtain. It’s not possible to achieve the best outcomes by only relying on a general model. But give it a few years, and this will change. Models will become so general that with only a small sample from the domain you want to apply them to, they’ll quickly specialize for that application.

Most AI engineers will be replaced by AI sooner than software developers, who will still be needed to integrate these AI models into applications.
And I don’t think AI agents can steal their jobs completely — not yet, at least. Someone still has to use the AI to build the application they want, and that person needs the knowledge to ensure the AI is doing the right thing in the best way possible. This isn’t something AI can fully replace, since agents need a user, and that user has to know what they’re building.

But this is the topic for a different post. :)