Building a modern API using PowerShell
Powershell may not be the first language people think of when they need to build an API or a modern web app, but not only is it possible, but for a low to mid-usage system it's actually pretty easy to set up and roll out. It helps that I am using a framework designed to do it though. So let's start there, the framework: Pode. Pode is an extremely powerful module for PowerShell written by Badgerati and is available on his GitHub here: https://github.com/Badgerati/Pode.
I haven't met Badgerati, nor do I even know his real name, but after using Pode to build a few systems I consider him a genius. The way the module works in its simplicity is outstanding. The documentation for the basics is all very easy to follow, however, some of the more advanced concepts could use an 'Explain it like I'm a 5th grader' makeover.
Professionally I've only used a tiny fraction of what Pode can do, and in my personal projects not much more than that. In addition to Pode for back-end APIs there is a companion piece for front-end development, Pode.Web, but since that has not yet reached 1.0 maturity I can't recommend it for professional use yet. Still, I encourage you to take a deeper dive into the Pode ecosystem should you need to build a modern web app.
Before I get further into this, let's set up my use case. Presently I have a Windows application that talks to an on-premise database, and we want to transition this one application, which is a small subset of what the total DB is used for, to a web-based app. So the powers that be started designing this new web-based app in AWS and we hit a snag where the new app still needed to make infrequent, but real-time queries to other things in an on-prem database that isn't undergoing a digital transformation just yet.
An easy way to fix that, set up a VPN between AWS and on-premise. Yeah, that could be done, however, we decided that would not be as secure as having a dedicated API in between with controls and authentication to make both the AWS environment and the on-premise environment as secure as possible. Specifically we wanted to make sure that if one were to get compromised that the other would not be. So this API will run on an internal server that can query our SQL server and be accessed only by the AWS app passing an authentication token between them that we can validate.
The basics of Pode is that you can run in a single file the entirety of the web service/API, however, anything larger than a tiny API benefits from some structure. Pode has a nice command to automatically import routes from other PS1 files located in a folder called 'routes' that exists in the folder with your main API file. If you are also presenting a web page for this API, you may want/need to split out folders for CSS or JavaScript as well, which would be presented in a 'public' folder that Pode makes available. If you have HTML pages they could also be put into the public folder.
Best practice, I organize my folder structure like this:
>Public
>Script
>Style
>Fonts
>Images
>Routes
>Beta
>V1
api.ps1
functions.ps1
start.ps1
In my particular use case, I did not need a public folder, but have done this setup in other projects where I built a full front end and it works well to keep things organized. I also include a functions.ps1 file which defines any global functions that my routes may need to use. The start .ps1 file is used to start the Pode server, usually for development. Since Pode has to run in admin mode to be able to open a port, the start file has a little self-elevating code:
$newProcess = new-object System.Diagnostics.ProcessStartInfo "PowerShell"
$newProcess.Arguments =@"
start-podeserver -filepath $pathToApiPS1
"@
$newProcess.Verb = "runas";
# Start the new process
[System.Diagnostics.Process]::Start($newProcess);
Moving on to the API.ps1 file:
In production, Pode would be run either as a scheduled task or using a utility like NSSM to run as a service. Usually, that means you won't see the session window, but during testing, it is extremely helpful to be able to write errors out to the host window, as such, the first thing I always do when starting my pode API is to write the current time.
write-podehost $(get-date)
Yes, Pode does overwrite write-host, and best practice, use the Pode functions.
During development and using a regular PowerShell session Pode allows you to restart the Pode server with a keystroke, ctrl-r. Because our routes are distinct from the API.ps1 it will reload any changes made to the route files, but it will not reflect changes made to the base API.ps1 file; you'll need to terminate and restart Pode if you have made changes there. It's specifically for this reason that I write the time, as I can check that time vs the timestamp on the saved file because sometimes when I am writing code I forget to hit save before I restart the service...
I also recommend setting up logging of some sort, whether you want to log to a file or the event viewer is up to you. In this project, I opted for logging to the event viewer.
#In order to write to the event log, we might need to create an event log source
try{[System.Diagnostics.EventLog]::CreateEventSource('Pode', 'Application')}catch{}
#enable writing to the eventviewer
New-PodeLoggingMethod -EventViewer | Enable-PodeErrorLogging -Levels @('error','warning','informational')
#note in the event viewer we have started the Pode API.
[System.Exception]::new("Pode started at $(get-date)")|write-podeerrorlog -Level informational
Pode requires us to set up an endpoint, essentially what it is listening on. For dev purposes, setting up an HTTP listener on localhost is acceptable, but eventually, when this is exposed to the internet you'll want to use HTTPS and a proper certificate. Note, that you can also use nonstandard ports when setting up an endpoint, so you can have multiple APIs on different ports running on the same machine.
#Uncomment for using localhost
#add-podeendpoint -address localhost -port 80 -protocol Http
add-podeendpoint -address * -port 443 -protocol Https -certificatename '*.<yourdomain>.com' -CertificateStorelocation LocalMachine
Because the user of this API is an app in the AWS cloud, vs this being a self-contained web app with the front end also presented by Pode, we need to set some security headers. If we did not, web browsers would balk at the API not being in the same place and disallow the front end communicating to the back end. It would be a different story however if the AWS front end was running a script to scrape info periodically, but this runs live as the user interacts.
Pode allows us to set some middleware to do this, there are also functions to manipulate the security headers itself, but this is one of those areas where the Pode documentation gets rather complex, and the way I have presented it is what worked for me to get around the CORS error we had while this was being developed.
Add-PodeMiddleware -Name 'MandatoryAuthorizationHeader' -ScriptBlock {
Add-PodeHeader -Name 'Access-Control-Allow-Origin' -Value '*' #use your specific frontend URL here
#if your API needs to use POST, PATCH or DELETE (or any other methods) add them below.
Add-PodeHeader -Name 'Access-Control-Allow-Methods' -Value 'GET, OPTIONS'
Add-PodeHeader -Name 'Access-Control-Allow-Headers' -Value 'Content-Type,Authorization'
return $true
}
# Allow the option method in each route
Add-PodeRoute -Method Options -Path * -ScriptBlock {
return $true
}
Next, we will add our authentication for the API. In my case, AWS will be passing a bearer token, which I will need to validate with our front-end app. I'll interject here that Pode is really powerful regarding authentication schemes, having built-in methods for Azure AD, Active Directory, OAuth2, etc. The latest version of Pode even allows the use of multiple authentication schemes in various configurations on the same route.
New-PodeAuthScheme -Bearer | Add-PodeAuth -Name 'bearer' -Sessionless -ScriptBlock {
param($token)
#check if the token is valid, and get user. If valid return user otherwise return null.
$tokenobj=convertfrom-podejwt -token $token -IgnoreSignature
#write-podehost $tokenobj
$AuthURI='<AWS front end auth url>'
$authheader=@{authorization="Bearer $token"}
$response=invoke-restmethod -method get -uri $authuri -headers $authheader
#write-podehost $response
if($null -ne $response.error){return $null}
return @{ User = $tokenobj.username }
}
After all that, I'll import the routes placed in my routes folder and any global functions the routes will use from my functions file. If I had any routes I wanted to specify in my API.ps1 file I'd also do that here, and finally enable the OpenAPI docs, more on that later.
#read in all the files in the routes folders
use-poderoutes
#load the functions script
use-podescript -path $pathToFunctions
#enable openapi
Enable-podeopenapi -path '/docs/openapi' -title 'API Docs' -version 1.0 -routefilter '/api/*'
enable-podeopenapiviewer -type ReDoc -path '/docs/redoc'
enable-podeopenapiviewer -type swagger -path '/docs/swagger'
At the bottom of this article, I've posted the full example code for api.ps1.
For defining a route I like to use a template. In the template, we first will define metadata about the route, then define the route itself, which we will then pass to the OpenAPI handling functions to have Pode automatically build our OpenAPI documentation.
The metadata will start by defining where to access the route, IE myurl.mydomain.com<$routepath>. Everything else is only going to be used by the OpenAPI system and doesn't have any bearing on how Pode interprets the route, but for documenting to users how to use the Route it is important to present what parameters exist, and what valid and error responses they can expect.
#Define Route Meta
$RoutePath = 'v1/customer'
$RouteSummary = 'Gets customer info'
$RouteTags = 'Customer'
$routeDeprecated=$false
$routeparameters=@(
(New-PodeOAIntProperty -Name 'custnum' -description 'Limits returned results to a given customer number' | ConvertTo-PodeOAParameter -In Query ),
(New-PodeOAboolProperty -Name 'echo' -Description 'Will echo the query to server log. Useful for troubleshooting' | ConvertTo-PodeOAParameter -In Query)
)
$route200Response=@{
statuscode=200
description= 'A JSON array of customers'
contentschemas=@{
'application/json' = (New-PodeOAObjectProperty -array -Properties @(
(New-PodeOAIntProperty -Name 'Custnum' -description "the ID number of the customer"),
(New-PodeOAStringProperty -Name 'CustName' -description "Customer Name"),
(New-PodeOAStringProperty -Name 'Custurl')
))
}
}
$route404Response=@{
statuscode=404
description='No customers found'
}
$route500Response=@{
statuscode=500
description='Not Authorized'
}
#end Define Route Meta
The beginning of the actual route is defined as such:
add-poderoute -method get -path $routepath -Authentication 'bearer' -scriptblock{
#begin route logic
$query=$webevent.Query
The $webevent variable is automatically populated by Pode with any query parameters or anything else from the web event. In other use cases, I pass a bit of JSON data using the PUT method to write data back.
The route runs some SQL commands against the SQL server depending on if the customer number was supplied and logs things if the Echo parameter was supplied and then ends by returning the result, if any. The end of the add-poderoute function is then piped to various OpenAPI functions.
write-podetextresponse -value (convertto-json $($dataset|select-object custnum,name, custurl))
#end route logic
} -passthru|
set-podeoarouteinfo -summary $routesummary -tags $routetags -deprecated:$routeDeprecated -passthru|
set-podeoarequest -parameters $routeparameters -requestbody $routerequestbody -passthru|
add-podeoaresponse @route200response -passthru|
add-podeoaresponse @route404response -passthru |
add-podeoaresponse @route500response
After the first route is set up, I'll clone it to build the other routes, making sure that the $routepath is changed for each route, as those must be unique.
When everything is done, we can start the Pode server and browse to the built-in URLs for OpenAPI using Swagger or ReDoc (I find I like Swagger a little better) which you can present to the users of the API. A functional API in only a few hundred lines of code complete with built-in documentation. My team who was building the AWS side of the app was impressed with the speed with which I was able to implement this, and the speed at which I can update it to have additional routes.
The finishing touches on this project would be getting it to be load-balanced running on multiple servers, and persistent across a server reboot, but that may be better suited for another article.
A word of caution though: PowerShell is "slow" compared to other languages. It's a scripting language so this may not be the best choice if you have millions of requests an hour. In the places I've used Pode at most I've had ~40 simultaneous users and performance has been great. I haven't hit the levels where performance degrades yet, and Pode does have built-in multithreading to mitigate that, but at some number of requests, I'm guessing some other language would be the better option.
While the two files in my example are not fully functional, since I omit my URLs and other project-unique info, it's pretty close to being fully functional and hopefully if you are working on a similar project you can glean some insight from these. The API.ps1 and get-customer.ps1 files are below.
{ #full api.ps1 example
write-podehost $(get-date)
#In order to write to the event log, we might need to create an event log source
try{[System.Diagnostics.EventLog]::CreateEventSource('Pode', 'Application')}catch{}
#enable writing to the eventviewer
New-PodeLoggingMethod -EventViewer | Enable-PodeErrorLogging -Levels @('error','warning','informational')
#note in the event viewer we have started the Pode API.
[System.Exception]::new("Pode started at $(get-date)")|write-podeerrorlog -Level informational
#Uncomment for using localhost
#add-podeendpoint -address localhost -port 80 -protocol Http
add-podeendpoint -address * -port 443 -protocol Https -certificatename '*.<yourdomain>.com' -CertificateStorelocation LocalMachine
Add-PodeMiddleware -Name 'MandatoryAuthorizationHeader' -ScriptBlock {
Add-PodeHeader -Name 'Access-Control-Allow-Origin' -Value '*' #use your specific frontend URL here
#if your API needs to use POST, PATCH or DELETE (or any other methods) add them below.
Add-PodeHeader -Name 'Access-Control-Allow-Methods' -Value 'GET, OPTIONS'
Add-PodeHeader -Name 'Access-Control-Allow-Headers' -Value 'Content-Type,Authorization'
return $true
}
# Allow the option method in each route
Add-PodeRoute -Method Options -Path * -ScriptBlock {
return $true
}
New-PodeAuthScheme -Bearer | Add-PodeAuth -Name 'bearer' -Sessionless -ScriptBlock {
param($token)
#check if the token is valid, and get user. If valid return user otherwise return null.
$tokenobj=convertfrom-podejwt -token $token -IgnoreSignature
#write-podehost $tokenobj
$AuthURI='<AWS front end auth url>'
$authheader=@{authorization="Bearer $token"}
$response=invoke-restmethod -method get -uri $authuri -headers $authheader
#write-podehost $response
if($null -ne $response.error){return $null}
return @{ User = $tokenobj.username }
}
#read in all the files in the routes folders
use-poderoutes
#load the functions script
use-podescript -path $pathToFunctions
#enable openapi
Enable-podeopenapi -path '/docs/openapi' -title 'API Docs' -version 1.0 -routefilter '/api/*'
enable-podeopenapiviewer -type ReDoc -path '/docs/redoc'
enable-podeopenapiviewer -type swagger -path '/docs/swagger'
}
#get-customer.ps1
#Define Route Meta
$RoutePath = 'v1/customer'
$RouteSummary = 'Gets customer info'
$RouteTags = 'Customer'
$routeDeprecated=$false
$routeparameters=@(
(New-PodeOAIntProperty -Name 'custnum' -description 'Limits returned results to a given customer number' | ConvertTo-PodeOAParameter -In Query ),
(New-PodeOAboolProperty -Name 'echo' -Description 'Will echo the query to server log. Useful for troubleshooting' | ConvertTo-PodeOAParameter -In Query)
)
$route200Response=@{
statuscode=200
description= 'A JSON array of customers'
contentschemas=@{
'application/json' = (New-PodeOAObjectProperty -array -Properties @(
(New-PodeOAIntProperty -Name 'Custnum' -description "the ID number of the customer"),
(New-PodeOAStringProperty -Name 'CustName' -description "Customer Name"),
(New-PodeOAStringProperty -Name 'custurl')
))
}
}
$route404Response=@{
statuscode=404
description='No customers found'
}
$route500Response=@{
statuscode=500
description='Not Authorized'
}
#end Define Route Meta
add-poderoute -method get -path $routepath -Authentication 'bearer' -scriptblock{
#begin route logic
$query=$webevent.Query
#$db=(get-podeconfig).sqlsettings.database
$ds=(get-podeconfig).sqlsettings.datasource
$un=(get-podeconfig).sqlsettings.username
$pw=(get-podeconfig).sqlsettings.password
if($query.echo){[System.Exception]::new("Customer API queried for $(convertto-json $query)")|write-podeerrorlog -Level informational}
if($null -eq $query.custnum){
$sqlcommand="select custnum, name, custurl from crm_customers where custnum is not null"
$dataset=invoke-sqlcmd -query $sqlcommand -serverinstance $ds -username $un -password $pw -OutputSqlErrors $true -IncludeSqlUserErrors -TrustServerCertificate
}
else{
$sqlcommand="select custnum, name, custurl from crm_customers where custnum = `$(var1) "
$var1=$query.custnum
$dataset=invoke-sqlcmd -query $sqlcommand -serverinstance $ds -username $un -password $pw -OutputSqlErrors $true -IncludeSqlUserErrors -variable @("var1='$var1'") -TrustServerCertificate
}
if($query.echo){[System.Exception]::new("$(Convertto-json $result)")}
write-podetextresponse -value (convertto-json $($dataset|select-object custnum,name, sdms_url))
#end route logic
} -passthru|
set-podeoarouteinfo -summary $routesummary -tags $routetags -deprecated:$routeDeprecated -passthru|
set-podeoarequest -parameters $routeparameters -requestbody $routerequestbody -passthru|
add-podeoaresponse @route200response -passthru|
add-podeoaresponse @route404response -passthru |
add-podeoaresponse @route500response