HTTP/HTTPS Requests via Invoke-WebRequest PowerShell Cmdlet

The Invoke-WebRequest cmdlet allows you to send HTTP/HTTPS/FTP requests, receive and process responses, and return sets of elements from an HTML page (links, images, forms, etc.). The cmdlet is available starting from PowerShell 3.0. There are two aliases available for the Invoke-WebRequest cmdlet, iwk and wget. In this article, we’ll look at how to use this cmdlet to send simple requests to HTML documents and process the results.

The simplest PowerShell request for an HTTP website page looks like this:

Invoke-WebRequest -Uri theitbros.com

invoke webrequest powershell

Hint. Sometimes you can face “Invoke-WebRequest: The request was aborted: Could not create SSL/TLS secure channel” error when performing a request to an HTTPS site. Most likely the problem is that PowerShell uses TLS 1.0 by default to connect, and the target site requires TLS 1.2. To change the list of available security protocols for PowerShell connection, run the commands:

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls, [Net.SecurityProtocolType]::Tls11, [Net.SecurityProtocolType]::Tls12, [Net.SecurityProtocolType]::Ssl3

[Net.ServicePointManager]::SecurityProtocol = "Tls, Tls11, Tls12, Ssl3"

The Invoke-WebRequest cmdlet allows you to parse HTML page content as a set of attributes and properties. For example, you can get a list of links on a page:

$Site = "theitbros.com"

$HttpContent = Invoke-WebRequest -Uri $Site

$HttpContent.Links

Or a list of images on the page:

$HttpContent.Images

$HttpContent.Images.src

You can get the Title from a web page:

$($HttpContent.ParsedHtml.getElementsByTagName('title')).innertext;

You can check the HTTP response status of the page:

$HttpContent.StatusCode

powershell web request

In this example, the HTML page returned an HTTP status 200. This means that the request was successful and the webpage is available.

In order to get basic information about the server, cookies, and other headers:

$HttpContent.baseresponse

powershell webrequest

IsMutuallyAuthenticated : False

Cookies : {}

Headers : {Transfer-Encoding, Connection, Vary, Cache-Control…}

SupportsHeaders : True

ContentLength : -1

ContentEncoding :

ContentType : text/html; charset=UTF-8

CharacterSet : UTF-8

Server : nginx/1.16.1

LastModified : 9/24/2021 7:52:26 PM

StatusCode : OK

StatusDescription : OK

ProtocolVersion : 1.1

ResponseUri : http://theitbros.com/

Method : GET

IsFromCache : False

As you can see, you can get information about any element on a web page. All items are parsed as PowerShell objects, which is very convenient.

A complete list of all properties of an HTML page can be displayed as follows:

$HttpContent | Get-Member

The following attributes and properties are available:

  • Dispose
  • Equals
  • GetHashCode
  • GetType
  • ToString
  • AllElements
  • BaseResponse
  • Content
  • Forms
  • Headers
  • Images
  • InputFields
  • Links
  • ParsedHtml
  • RawContent
  • RawContentLength
  • RawContentStream
  • Scripts
  • StatusCode
  • StatusDescription

The HTML content of a web document itself can be accessed by two properties:

$HttpContent.Content

$HttpContent.RawContent

The Invoke-WebRequest cmdlet allows you to set your own User-Agent value for the connection:

Invoke-WebRequest -Uri $uri -UserAgent 'Mozilla/5.0'

Or you can set the user agent properties via HTTP headers:

$headers = @{

'method' = 'GET'

'cache-control' = 'no-cache'

'user-agent' = 'Mozilla/5.0 (Windows NT 5.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36'}

Invoke-WebRequest -Uri $uri -Headers $headers

If you need to authenticate to access the web page, you can use the following commands:

Basic Authentication:

$creds=Get-Credentials

$Site = "theitbros.com"

$HttpContent = Invoke-WebRequest -Uri $Site –Credential $creds

To perform NTLM/Kerberos authentication under the current user credentials:

wget $Site -UseDefaultCredentials

Authentication using a certificate (you need to specify it Thumbprint):

wget $Site -CertificateThumbprint 0C64E8B1113E86C76EF1CCA458E5B2A693946E90

Hint. The Invoke-WebRequest cmdlet allows you to download any file available via FTP/HTTP(s). To do this, use the OutFile parameter:

Invoke-WebRequest -Uri 'https://theitbros.com/media/template.zip' -OutFile C:\downloads\itbros_template.zip.jpg
I enjoy technology and developing websites. Since 2012 I'm running a few of my own websites, and share useful content on gadgets, PC administration and website promotion.
Cyril Kardashevsky

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.