There are major differences in the way different parsing libraries and tools handle Uniform Resource Locator (URLs), new research showed.

That becomes very important because URLs are in many ways the hub of our digital lives, our link to critical services, news, entertainment, and much more. Therefore, any security vulnerabilities with how browsers, applications, and servers receive URL requests, parse them, and fetch requested resources could pose significant issues for users and harm trust in the Internet.

If you click on a URL or type it into a browser, you’re requesting a resource hosted somewhere online. As a result, some devices such as our browsers, applications, and servers must receive our URL, parse it into its uniform resource identifier (URI) components (e.g. hostname, path, etc.) and fetch the requested resource.

The syntax of URLs is complex, and although different libraries can parse them accurately, it is plausible for the same URL to be parsed differently by different libraries. The confusion in URL parsing can cause unexpected behavior in the software (e.g. web application), and could end up exploited by threat actors to cause denial-of-service conditions, information leaks, or possibly conduct remote code execution attacks, according to a research paper by Noam Moshe and Sharon Brizinov of Claroty’s Team82, in collaboration with Raul Onitza-Klugman and Kirill Efimov of Snyk’s research team.

Team82’s joint research with Snyk examined 16 URL parsing libraries, written in a variety of programming languages, and noticed inconsistencies with how each chooses to parse a given URL to its basic components. The researchers categorized the types of inconsistencies into five categories, and searched for problematic code flows in web applications and open source libraries that exposed a number of vulnerabilities.

Schneider Bold

As a result of their analysis, they were able to identify and categorize five different scenarios in which most URL parsers behaved unexpectedly:

Scheme Confusion: A confusion involving URLs with missing or malformed scheme

Slash Confusion: A confusion involving URLs containing an irregular number of slashes

Backslash Confusion: A confusion involving URLs containing backslashes (\)

URL-Encoded Data Confusion: A confusion involving URLs containing URL Encoded data

Scheme Mixup: A confusion involving parsing a URL belonging to a certain scheme without a scheme-specific parser

Researchers learned most of the eight vulnerabilities they found largely occurred for two reasons:

  1. Multiple parsers in use: Whether by design or an oversight, developers sometimes use more than one URL parsing library in projects. Because some libraries may parse the same URL differently, vulnerabilities could be introduced into the code.
  2. Specification incompatibility: Different parsing libraries are written according to different RFCs or URL specifications, which creates inconsistencies by design. This also leads to vulnerabilities because developers may not be familiar with the differences between URL specifications and their implications.

The eight vulnerabilities below have been patched except for those found in unsupported versions of Flask:

  1. Flask-security (Python, CVE-2021-23385)
  2. Flask-security-too (Python, CVE-2021-32618)
  3. Flask-User (Python, CVE-2021-23401)
  4. Flask-unchained (Python, CVE-2021-23393)
  5. Belledonne’s SIP Stack (C, CVE-2021-33056)
  6. Video.js (JavaScript, CVE-2021-23414)
  7. Nagios XI (PHP, CVE-2021-37352)
  8. Clearance (Ruby, CVE-2021-23435)

Click here to download the paper by Claroty’s Team82, in collaboration with Snyk’s research team.

Pin It on Pinterest

Share This