Jan 1, 2024

Forms in Hugo - Backend stuff

A little guide on how to implement Forms in Hugo. Part 1

BLZR

PersonalWebPrivacyCode

Forms in Hugo

2564 Words | Reading Time: 11 Minutes, 39 Seconds

31-12-2023 07:31 PM UTC

Every self-respecting blog needs a Contact section and a fleeting hope that some audience/reader will use that to contact them; which may as well steer their life into an uncharted territory. It is believed that having a Contact section in blog actually helps if readers cares so much that they will spare a couple of minutes to appreciate or rectify mistakes. In addition to that, a blog with a contact page is thought to do well in terms of SEO; for which I have no studies to cite to.

That being said, and having accepted that the contact form in my blog is as useless as Anne Frank’s drumset, let’s see how I managed to put together a living mess which I call a contact form.

Before, we jump any deeper, we need to know that Hugo generates a static webpage by default, so once a content is served, the server has no fucking clue on what you are doing with it. Sure, you can bring out creativity out of your ass to make it a WordPress clone and add a buttload of dynamic elements. But, if that’s the objective, you are better off using WordPress. Where was I? Yes, Hugo makes static pages, Sever has no clue yada-yada-yada…

This staticness (I learned that today) creates a problem that I cannot add a contact form and have a backend to gather data published to it. My site is generously hosted by free version of GitLab, any hope that I will have a dynamic backend is out of the window (which is ironic because I work as a backend engineer in my day job).

This is how I ended up in this place. Are you ready for a ride?

Yoda says, The sound of crickets I hear

Well, fuck me

Step 1. Getting backend ready

One thing I have learned about websites where front-end and backend doesn’t talk to one another, is to make backend first.

And for me that backend is….. 🥁 🥁 🥁 please…. It’s Telegram. Yup, that’s my backend. That is my NoSQL database.

Having a Telegram Bot

I have created a Telegram Bot. You can make your very own bot from BotFather. Keep the BOT_TOKEN noted down. Next thing is to get your Chat ID. Yes, there is a chatid associated to your Telegram account. If you can read Telegram Bot’s logs, else follow this article.

P.S. You can read bot logs by going to this URL : https://api.telegram.org/bot<BOT_TOKEN>/getUpdates

That is only interaction needed from Telegram as of now.

Making a Cloudflare Worker

Keeping in line with my cheap-ass-ness; and due to the fact that my domain’s DNS is Cloudflare, I will be using Cloudflare Worker. It is the same as AWS Lambda or Azure Functions, as it provides a microservice platform to run a simple script which will listen to the form output and forward it to Telegram bot.

To create a Cloudflare Worker, go to : https://dash.cloudflare.com/<ACCOUNT_ID>/workers-and-pages/create/workers/new to create a new worker application in your Cloudflare account. You could follow a tutorial, if you are lost in the maze of settings or have zero idea where to start. I found this article quite useful to help a newbie get started.

To get a feel on what the actual heck you’re doing, clone my repo to your local machine and we can follow along.

After cloning, you have to do two things
  1. Install node.js and npm. After that, you can install the dependencies.
  2. And you will have to create a file called wrangler.toml. Fill it up as follows:
1
2
3
4
5
6
7
8

name = "<NAME_OF_WORKER>"
main = "index.js"
compatibility_date = "2022-07-12"

[vars]
CHATID = "<CHAT_ID>"
ORIGIN_URI = "https://<YOUR_DOMAIN.COM>"
TELEGRAMTOKEN = "<BOT_TOKEN>"

We shall discuss index.js and it will be evident why we are using the variables in wrangler.toml

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

const corsHeaders = {
    "Access-Control-Allow-Origin": ORIGIN_URI,
    "Access-Control-Allow-Methods": "POST, OPTIONS",
    "Access-Control-Allow-Headers": "Content-Type",
}

addEventListener("fetch", event => {
    event.respondWith(handleRequest(event.request))
})

async function TG(BODY) {
    const bot_url = `https://api.telegram.org/bot${TELEGRAMTOKEN}/sendMessage`;
    const messageSend = JSON.stringify({ "chat_id": CHATID, "text": BODY });
    try {
        let response = fetch(bot_url, { method: 'POST', headers: { "Content-Type": "application/json" }, body: messageSend });
        return await response;
    } catch (error) {
        console.log(error);
    }
}
async function UPDATE(request) {

    const obj = await request.json();

    const BODY = `
    
    from: blzr.sbs            
    =================================
    Message : 
    ${obj.MSG}
  
    =================================
    User Agent = ${obj.UA}
    Language = ${obj.LANG}
    SYS DT = ${obj.DT}, ${obj.ZONE}
    
    =================================
    IP = ${request.headers.get('CF-Connecting-IP')}
    CF Edge = ${request.cf.colo}
    LOCATION = ${request.cf.city}, ${request.cf.country}, ${request.cf.postalCode}, ${request.cf.region}
    Location = https://www.google.com/maps/place/${request.cf.latitude}+${request.cf.longitude}
    Timezone = ${request.cf.timezone}
    ISP = ${request.cf.asOrganization}
    ====================================`;

    await TG(BODY);
}
async function ACTIVITY(request) {
    const BODY = `
    
    ACTIVITY LOG
    ------------------------------------
    ------------------------------------
    IP = ${request.headers.get('CF-Connecting-IP')}
    CF-RAY = ${request.headers.get('cf-ray')}
    User-Agent = ${request.headers.get('user-agent')}
    Colo = ${request.cf.colo}
    Country = ${request.cf.country}
    City = ${request.cf.city}
    Location = https://www.google.com/maps/place/${request.cf.latitude}+${request.cf.longitude}
    ASN = ${request.cf.asn}
    PostalCode = ${request.cf.postalCode}
    Region = ${request.cf.region}
    Timezone = ${request.cf.timezone}
    Organization = ${request.cf.asOrganization}
    BODY = 
    ${JSON.stringify(await request.json(), null, 2)}
    ====================================`;

    await TG(BODY);
}

function handleOptions(request) {
    if (request.headers.get("Origin") !== null &&
        request.headers.get("Access-Control-Request-Method") !== null &&
        request.headers.get("Access-Control-Request-Headers") !== null) {
        return new Response(null, {
            headers: corsHeaders
        })
    } else {
        return new Response(null, {
            headers: {
                "Allow": "OPTIONS",
            }
        })
    }
}

async function handleRequest(request) {
    const url = new URL(request.url)

    if ((url.pathname === "/submit") && (request.method === "POST") && (request.headers.get('Origin') === ORIGIN_URI)) {
        await UPDATE(request);

        return new Response(JSON.stringify({ COMMENT: 'If you are seeing this, then we have a lot in common' }), {
            status: 200,
            headers: {
                "Content-Type": "application/json",
                ...corsHeaders,
            }
        })
    }
    else if (request.method === "OPTIONS") {
        return handleOptions(request)
    }
    else {
        await ACTIVITY(request);
        return new Response('492077696c6c206e6f742070726f6365737320746869732072657175657374', {
            headers: { 'content-type': 'text/html' },
            status: 405
        })
    }
}
CORS Headers

Let us see Lines 1 to 5 first, as this is most important, and I had spent most of my debugging time here.

1
2
3
4
5

const corsHeaders = {
    "Access-Control-Allow-Origin": ORIGIN_URI,
    "Access-Control-Allow-Methods": "POST, OPTIONS",
    "Access-Control-Allow-Headers": "Content-Type",
}

CORS or Cross Origin Request Sharing, is a 1“mechanism that allows a server to indicate any origins (domain, scheme, or port) other than its own from which a browser should permit loading resources. CORS also relies on a mechanism by which browsers make a “preflight” request to the server hosting the cross-origin resource, in order to check that the server will permit the actual request. In that preflight, the browser sends headers that indicate the HTTP method and headers that will be used in the actual request.”

In other words, this allows me to transfer data from my Hugo site to the worker, and stopping browser from complaining. It is a HTTP Header based system. I am allowing POST and OPTIONS methods from ORIGIN_URI (which we had set up in wrangler.toml) and accepting additional header of Content-Type to be sent to Worker.

Now, that’s done, we can move to the next part, which will run when we receive a data packet from the internet.

 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

async function handleRequest(request) {
    const url = new URL(request.url)

    if ((url.pathname === "/submit") && (request.method === "POST") && (request.headers.get('Origin') === ORIGIN_URI)) {
        await UPDATE(request);

        return new Response(JSON.stringify({ COMMENT: 'If you are seeing this, then we have a lot in common' }), {
            status: 200,
            headers: {
                "Content-Type": "application/json",
                ...corsHeaders,
            }
        })
    }
    else if (request.method === "OPTIONS") {
        return handleOptions(request)
    }
    else {
        await ACTIVITY(request);
        return new Response('492077696c6c206e6f742070726f6365737320746869732072657175657374', {
            headers: { 'content-type': 'text/html' },
            status: 405
        })
    }
}

We are running an asynchronous function, which will trigger every time there is a data packet. At first, we are getting the context, which we shall be denoted by request. This request contains everything for us to work with.

Then, we are checking who sent the data to the worker.

90

const url = new URL(request.url)

Next, I am checking for three things.

  1. Does this request arrive in /submit path
  2. Does this request have a POST HTTP Header
  3. Did this request originate from ORIGIN_URI
92

if ((url.pathname === "/submit") && (request.method === "POST") && (request.headers.get('Origin') === ORIGIN_URI)) {

If the above conditions are satisfiable, we are transferring control to process the data to UPDATE() function, and giving browser the response that Worker has received the request (with 200 response code).

 93
 94
 95
 96
 97
 98
 99
100
101

await UPDATE(request);

return new Response(JSON.stringify({ COMMENT: 'If you are seeing this, then we have a lot in common' }), {
    status: 200,
    headers: {
        "Content-Type": "application/json",
        ...corsHeaders,
    }
})

Now, if you have worked with sending form data from one computer to another over a network, have scratched your head on why it is not working and end up hating CORS even more; Yep, Browser sends a Preflight OPTIONS request to check if it can really send the data or not.

Basic OPTIONS know-how 2

The OPTIONS request mentioned in the introduction is a preflight request, which is part of the CORS (Cross-Origin Resource Sharing). CORS is a mechanism that provides configuration to configure access to shared resources. CORS applies when a webpage makes a request to another server other than its origin server, this could mean that either the domain, protocol, or port differs.

Using the request, the browser checks with the server whether the request is allowed. Only if the request is allowed, it’ll actually perform it.

We can perform a GET request without the need for a preflight request. However, the restrictions for POST requests are tighter. It means, for example, that we cannot send a JSON request without a preflight.

Thus, to handle OPTIONS, we need another check which invokes handleOptions() function.

103
104
105

else if (request.method === "OPTIONS") {
    return handleOptions(request)
}

If none is true, which usually happens when someone decides to send useless requests to the Worker, we invoke the else part. This return a response of 405 with a hash of some text which I forgot. If you are able to decode that, let me know.

106
107
108
109
110
111
112

else {
    await ACTIVITY(request);
    return new Response('492077696c6c206e6f742070726f6365737320746869732072657175657374', {
        headers: { 'content-type': 'text/html' },
        status: 405
    })
}

Function Explanation

Now, we should take a look into what each function does in this script. First and foremost, we should check the handleOptions, as it is the first function to be invoked when sending a form response. Here, I’m doing a basic check if there is a valid Origin point, has proper method and headers. If all three conditions are satisfied, I’m sending the CORS headers, which we have discussed before. Else, I am sending allowed header that I’m accepting only OPTIONS, which helps in reducing useless spam.

73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

function handleOptions(request) {
    if (request.headers.get("Origin") !== null &&
        request.headers.get("Access-Control-Request-Method") !== null &&
        request.headers.get("Access-Control-Request-Headers") !== null) {
        return new Response(null, {
            headers: corsHeaders
        })
    } else {
        return new Response(null, {
            headers: {
                "Allow": "OPTIONS",
            }
        })
    }
}

The next part is when someone decides to spam me with useless requests. It is important that it should be logged so that I could put them in blacklist. I’m creating an activity log of the sender’s request. I am logging the following information:

  1. IP Address of Request Origin
  2. Cloudflare Ray ID - it is an identifier given to every request that goes through Cloudflare
  3. User Agent
  4. The Closest Cloudflare Datacenter which got this request
  5. Country from which Request originated
  6. City from which Request originated
  7. Approximate geographical location, using latitude & longitude, and using Google Maps for showing me on the map
  8. ASN of the incoming request
  9. Postal code of the incoming request
  10. Timezone from where the request was generated
  11. ISP used for sending the request
  12. Any content in the request in JSON format
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

async function ACTIVITY(request) {
    const BODY = `
    
    ACTIVITY LOG
    ------------------------------------
    ------------------------------------
    IP = ${request.headers.get('CF-Connecting-IP')}
    CF-RAY = ${request.headers.get('cf-ray')}
    User-Agent = ${request.headers.get('user-agent')}
    Colo = ${request.cf.colo}
    Country = ${request.cf.country}
    City = ${request.cf.city}
    Location = https://www.google.com/maps/place/${request.cf.latitude}+${request.cf.longitude}
    ASN = ${request.cf.asn}
    PostalCode = ${request.cf.postalCode}
    Region = ${request.cf.region}
    Timezone = ${request.cf.timezone}
    Organization = ${request.cf.asOrganization}
    BODY = 
    ${JSON.stringify(await request.json(), null, 2)}
    ====================================`;

    await TG(BODY);
}

Next is UPDATE function which handles all the legitimate request coming from the form. I’m first converting the request to JSON for easier parsing. From the JSON Object, I am extracting:

  1. Message itself
  2. User agent of the browser
  3. Default language set in browser
  4. System Date and Time with Timezone
  5. IP Address
  6. Edge location
  7. City, Country, Postal code, region, latitude, longitude of the request origination
  8. Timezone provided by Cloudflare
  9. ISP Name
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

async function UPDATE(request) {

    const obj = await request.json();

    const BODY = `
    
    from: blzr.sbs            
    =================================
    Message : 
    ${obj.MSG}
  
    =================================
    User Agent = ${obj.UA}
    Language = ${obj.LANG}
    SYS DT = ${obj.DT}, ${obj.ZONE}
    
    =================================
    IP = ${request.headers.get('CF-Connecting-IP')}
    CF Edge = ${request.cf.colo}
    LOCATION = ${request.cf.city}, ${request.cf.country}, ${request.cf.postalCode}, ${request.cf.region}
    Location = https://www.google.com/maps/place/${request.cf.latitude}+${request.cf.longitude}
    Timezone = ${request.cf.timezone}
    ISP = ${request.cf.asOrganization}
    ====================================`;

    await TG(BODY);
}

The most important is TG function, which actually pushes the contents from Worker to Telegram through our already established Bot. When this function is invoked, it has a BODY for it to send out. Thus, it starts by fetching TELEGRAMTOKEN & CHATID from wrangler.toml and invokes a POST request to api.telegram.org using CHATID to determine where to send the payload to. If it encounters an error, it is automatically logged in Worker logs.

11
12
13
14
15
16
17
18
19
20

async function TG(BODY) {
    const bot_url = `https://api.telegram.org/bot${TELEGRAMTOKEN}/sendMessage`;
    const messageSend = JSON.stringify({ "chat_id": CHATID, "text": BODY });
    try {
        let response = fetch(bot_url, { method: 'POST', headers: { "Content-Type": "application/json" }, body: messageSend });
        return await response;
    } catch (error) {
        console.log(error);
    }
}

This article is long as it is. We will look into what should you do in Hugo itself to make contact form actually work. That part will be easy, I promise.

  1. https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS ↩︎

  2. https://www.baeldung.com/cs/why-options-request-sent ↩︎

See Also