Mirror a site behind a login or a CAPTCHA

Some sites show their pages only after you log in, or only after you pass a check such as a CAPTCHA. HTTrack cannot fill in a login form and cannot solve a CAPTCHA. But it can use the session of a browser where you already logged in, so the site accepts HTTrack as that browser. Since version 3.51, WinHTTrack, WebHTTrack and the Android app each have a button that does this in a few clicks.

When to use this method

Use it when one of these is true:

• The site has a login form, a single sign-on page (SSO), or a second step such as a code from your phone (2FA).
• The site shows a CAPTCHA, or a "checking your browser" page, before the content.
• The mirror contains the login page instead of the pages that you see in your browser.
• HTTrack gets "403 Forbidden", or a redirect to the login page, for pages that open in your browser.
• HTTrack stops with "MIRROR ERROR", and says that the mirror is empty, but the site opens in your browser.

Some sites ask for a user name and a password in a small browser window (HTTP Basic authentication). For these sites, you do not need this method. Put the user name and the password in the address, for example https://user:password@example.com/. See the FAQ.

WinHTTrack: log in with the built-in browser

1. Start a new project. On the page with the address box, click "Add URL...".
2. Type the address of the site, then click "Log in with a browser...". A browser window opens on this address.
3. Log in to the site as usual. If there is a CAPTCHA, solve it. Then go to the first page that you want to copy, and click "Done".
4. WinHTTrack tells you how many cookies it collected. The address box now shows the page where the browser stopped. Click "OK" to keep the login, or "Cancel" to discard it.

If the browser stops on another site than the one you started from, WinHTTrack warns you. Check the address before you go on.

The button needs the Microsoft Edge WebView2 Runtime, which Windows 10 and 11 already have. It does not work on Windows 7. When the button is greyed out, its tooltip tells you why.

WebHTTrack: paste the request of your browser

WebHTTrack runs in your browser, so it can take the login from that browser.

1. Log in to the site in another tab of the same browser.
2. Press F12 to open the developer tools, and open the "Network" tab. Reload the page.
3. Right-click the first request in the list (the page itself), and choose "Copy as cURL (bash)". Firefox calls it "Copy as cURL", or "Copy as cURL (POSIX)" on Windows.
4. In WebHTTrack, on the page where you type the addresses, click "Log in with a browser...". Paste the text and click "Go". Check the number of cookies, then click "OK".

A cookies.txt file works too: type the site address first in the address box, then paste the content of the file.

Android: log in inside the app

1. Start a new project. Give it a name, and type the address of the site. The address must start with https://.
2. Tap "Log in with a browser", under the address field.
3. Log in to the site as usual, then tap "Use this login". Google accounts do not work in this window.

The app confirms how many cookies it saved. It also sets 2 connections and skips logout links, so that your login lasts.

The app keeps the login in its private storage, so you never see a file. To delete it, tap "Forget the saved login" in the menu of the login window, or delete the project. "Forget the saved login" also signs you out of all the sites in the browser of the app.

What HTTrack keeps, and what to do next

WinHTTrack and WebHTTrack save the login in the project folder, in a file named after the site, for example example.com.hts. It holds the cookies and the User-Agent of the browser. They also add #httrack-bundle=example.com to the address, so the mirror uses this login. HTTrack sends the login only to this site and its subdomains.

Then start the mirror. In WinHTTrack and WebHTTrack, two settings protect your session:

• In "Set options", on the "Flow Control" tab, set "N# connections" to 2. Many fast requests from one logged-in user look like abuse, and the site can then end your session.
• On the "Scan Rules" tab, add -*logout*, -*signout* and -*sign-out*. A visit to the logout link ends your session.

Without the button: export the cookies by hand

Use these steps with an older version of HTTrack, or when the button cannot log in.

Step 1: export the cookies

1. Log in to the site with your usual browser. If there is a CAPTCHA, solve it. Then open a page that you want to copy, to make sure that you are logged in.
2. Save the cookies of this site to a cookies file named cookies.txt, in the Netscape format. Use a browser extension:
   • Chrome, Edge or Brave: install the extension "Get cookies.txt LOCALLY". It is open source. It writes the file on your computer and sends nothing to other computers. Open the site, click the extension, and export only the cookies of this site.
   • Firefox: install the extension "cookies.txt" or "Get cookies.txt LOCALLY". Export the cookies of the site in the same way.

Most session cookies are marked "HttpOnly" (not readable by scripts). In a cookies file, some tools (curl, yt-dlp) start the lines of these cookies with #HttpOnly_. HTTrack versions newer than 3.50.5 read these lines. Older versions treat them as comments and ignore them, so the login does not work. With an older version, remove #HttpOnly_ from the start of these lines.

Step 2: find the User-Agent of your browser

The User-Agent is the text that tells the site which browser it talks to. Some sites accept a cookie only from the browser that received it. Use the same browser that you logged in with:

• Firefox: type about:support in the address bar. Copy the "User Agent" line in the "Application Basics" section.
• Chrome or Edge: type chrome://version (or edge://version) in the address bar. Copy the "User agent" line.
• All browsers: press F12 to open the developer tools, and open the "Console" tab. Type navigator.userAgent and press Enter.

Copy the full text, with no change. It starts with Mozilla/5.0.

Step 3: run WinHTTrack or WebHTTrack

Start a new project and type the site address. Then click "Set options":

• On the "Spider" tab, type the path of the cookies file in "Load cookies from file". Keep "Accept cookies" selected.
• On the "Browser ID" tab, paste the User-Agent in "Identity".
• On the "Flow Control" tab, set "N# connections" to 2.
• On the "Scan Rules" tab, add -*logout*, -*signout* and -*sign-out*.

Or: run httrack on the command line

Instead of the interface, you can use the command line. Give the cookies file with --cookies-file (-%K) and the User-Agent with --user-agent (-F):

httrack https://example.com/members/ -O mysite --cookies-file cookies.txt --user-agent "PASTE-YOUR-USER-AGENT" --sockets=2 "-*logout*" "-*signout*" "-*sign-out*"

• --sockets=2 (-c2) opens only two connections at a time. Many fast requests from one logged-in user look like abuse. The site can then end your session.
• The filters "-*logout*", "-*signout*" and "-*sign-out*" stop HTTrack from following the logout link. A visit to that link ends your session. If the logout link of your site has a different name, add a filter for it. See the filter syntax page.

Limits

• Cookies expire. A session can end after some minutes or some days. If the mirror starts to contain login pages again, log in again, with the button or by exporting the cookies, and continue the mirror.
• Bot protection can still refuse HTTrack. Services such as Cloudflare tie their "clearance" cookie to your browser, so they can refuse HTTrack even with the right cookies and User-Agent. HTTrack writes a "Bot protection challenge" warning in its log when it sees a Cloudflare refusal.
• Respect the terms of the site. A login does not give you the right to copy everything. Read the terms of use of the site, and read Using HTTrack responsibly.