<!--
date: 2024-12-06
title: Mail-based Comment System
comment: true
-->

## Motivation

To get feedback, Comment System are essential to web pages.

Most of modern web pages use 3rd-party services like Disqus.
They are criticized by [ads] or trivial steps for sign-up/validation.
For my Point of View, even using Github issue is not a good idea.
I mean... You did make contents by yourself with markdown or some similar things, but why let others hold comments for you?

[CGI] is another solution if you can access a machine with public IP.
To show comments in your pages, you just need to transform HTTP Request into HTML.
For example, [this][yasuoka-calendar] one-line comment button would be easily implemented!
But consider spam or offensive content, a way to identify visitor still matters.

-><big>Of course I am talking about Mail!</big><-

For each comment, all you need to know are inside Mail header: sender title, mail address, datetime and SMTP server.
By SPF/DKIM/DMARC, receiver can nearly 100% confirm Mail is from a specific sender.


## Mailing List

Of course there is already a Comment System based on Mail: **[Mailing List]**.

Actually I am not a mailing-list user (though I did subscribe some lists of [FOSS] projects).
But thanks to public web page for archive[^1], I can realize how powerful it is.

User just need to finish subscription, then everyone can start a new thread by mail, and get replied by another mail.

But not everyone can build such this Comment System, you need to:

1. Be system admin to build related services
2. Tune HTTP web server to show archive
3. Afford traffic of SMTP for each new message in thread

This might be a little overkill for basic Comment System on simple web pages.


## MDA and .forward

If you can login as a normal user in Unix System (like [tilde.club]), `~/.forward` is what you need.

From decades ago, Unix use program called [MDA] to deliver mail/message[^2] to local user (Most of time it is [Sendmail]).
User can write their own `~/.forward` file to tell MDA where this mail should goes. For each entry in this file, it could be:

1. Another local user, for example: `alice`, `bob`
2. User in another host, for example: `alice@anotherhost.com`
3. Absolute pathname of a file, the mail would be appended to it. For example:  
   `/home/alice/mails`  
   Of course you can use `/dev/null` to drop any mail!
4. A command with leading char `|`, it would be executed with mail as STDIN. For example:  
   `"|/home/alice/mail-handler.sh arg1 arg2"`[^3]

With this approach, you can easily create a Common System based on mail:

1. Put a `mailto` link in your web page
2. Create a command to store incoming mail as comment in HTML format
3. Write an entry for this command in `.forward`
4. In your web page, display comments made by step.2

**Let's do it!**


## comment.sh

Let's build a bash script called `comment.sh` for all of these:

### 1. Check mail is for comment

I use pattern `Comment on page: <PATH_OF_WEB_PAGE>` in mail subject, to determine a mail should be stored as a comment. If it is not, script just return 0 safely:

```bash
# RFC 2822 use Carriage Return, remove it for safety
MAIL="$(tr -d '\r')"

# join multi-line header field value into one line
header="$(<<<"$MAIL" sed '/^$/ q; :a; N; s/\n\s\+//; ta')"

# just mail body
body="$(<<<"$MAIL" sed -n '/^$/,$ p' | sed '1d')"

# determine mail is for comment by pattern
pattern='^Subject: .*[cC]omment on page:? +(https?://[^/]+)?(/?[^ ]+).*$'
<<<"$header" grep -E "$pattern" >/dev/null || exit 0
```

### 2. Determine output and text process tool

For your HTTP routing, set `output_dir` to where HTML for comments locates.

Also, I use `markdown_bin` to specify command to process markdown text in mail body

```bash
output_dir=...
markdown_bin=...
```


### 3. Save each header field as variable

- For easier usage, save each field key as upper-case, and replace `-` with `_`
- Save datetime as rfc-3339 for more readable

```bash
# enable execute last command in pipe under current shell
shopt -s lastpipe; set +m;

# save each field of header into variables
echo "$header" | \
while read field value; do
  echo "$field" "$value" >>/tmp/header
  declare field=$(<<<$field tr [:lower:] [:upper:] | tr '-' '_' | tr -d ':')
  declare $field="${value}"
done

# formated datetime
DATE=${DATE:+$(date --rfc-3339 seconds --date "$DATE")}
```

### 4. Get path of output file

Based on mail subject, determine path of output. For example:  
if mail subject contains `Comment on page: /posts/comment-by-mail.html`,
then set output to `$output_dir/posts/comment-by-mail.comment.html`

```bash
path=...
output=$output_dir/${path}.comment.html
umask 022; mkdir -p $(dirname $output)
exec 1>$output
```


### 5. Get comment from mail body

By [RFC 1341], header with `Content-Type: multipart/*;` is separated by value of `boundary`.
Some mail client automatically sends mail with both **text part** and **html part**.
Here we extract text part if necessary:

```bash
printTextPart() {
  boundary=$1
  beginPat="\\|^--${boundary}\$|"
  endPat="\\|^--${boundary}(--)?\$|"
  sed -En "${beginPat},${endPat} { \@^Content-Type: text/plain@,$ {1,/^$/d; ${beginPat}d; p} }"
}

# check mail includes multiple part
boundary="$(<<<"$CONTENT_TYPE" sed -En 's/^.*boundary="?([^"]+)"?.*$/\1/p')"
if [ -n "${boundary}" ]; then
  # print mail part in MIME: text/plain
  body="$(<<<"$body" printTextPart ${boundary})"
fi
```


### 6. Write comment to output file

Time to write mail body to output file.
For Unix System like [tilde.club], set it readable for everyone:

```bash
umask 133
```

Then add basic `<style>` and `<ul>` for output file if

1. This mail is the first comment on page
2. Output file is illegal HTML

```bash
if [ ! -f $output ] || ! xmllint --html --nofixup-base-uris $output &>/dev/null; then
  <<-LAYOUT cat >$output
	<style>
	   ul {
	     ...
	   }
	  .comment-body {
	    ...
	  }
	  .replies {
	    ...
	  }
	</style>
	<ul>
	</ul>
	LAYOUT
fi
```

Finally, add a list item for comment:

```bash
# Append STDIN after line number with variable, or after first <ul> tag
<<-COMMENT sed -i "${line:-/<ul>/}r /dev/stdin" $output
	<li>

	<time datetime="${DATE}">${DATE}</time>
	<a href="mailto:${RECIPIENT}?subject=Comment on page: ${path}&in-reply-to=${MESSAGE_ID}">[reply]</a>

	<div class='comment-body'>
	$(<<<"${body}" ${markdown_bin})
	</div>

	<details class="replies" open="true">
	<summary>replies</summary>
	<ul>
	<!-- ${MESSAGE_ID} -->
	</ul>
	</details>

	</li>
COMMENT
```

Here we use:

0. Variable `$line` to insert this comment into output file
1. Variables comes from mail header, like `$DATE` or `$RECIPIENT`
2. An anchor element to reply to this comment (with `Message-ID` in mail header)
3. Command `$markdown_bin` to render HTML from mail body
4. Use `<details>` to create a foldable part for replies
5. In block of replies, add value of `Message-ID` as marker.
   It could be used to locate where to put other comments as replies

If visitor click the anchor element to reply, then mail client would prepare a new mail with header contains `In-Reply-To` with `$Message-ID`

With step.5, we can get `$line` number for new mail (which reply to this mail) by marker:

```bash
# get $line by field "In-Reply-To"
if [ -n "${IN_REPLY_TO}" ]; then
  line=$(grep -n "^<!-- ${IN_REPLY_TO} -->$" $output | cut -d':' -f1)
fi
```

### 7. ~/.forward

To co-work with MDA, just put this script into `~/.forward`:

```bash
# Inside ~/.forward

alice
|"/<PATH>/<TO>/comment.sh --output /srv/http"
```


## Set web page

To include HTML file generated by `comment.sh`, we can use `<object>` or `<iframe>`:

```html
<object type="text/html" data="/mypage.comment.html"></object>
```

Since height of `<object>` grows by number of comments, comment block would be fixed sized with scroll bar. Here we can use `ResizeObserver` to set height by document inside:
```html
<object ... onload="observeResize(this)"></object>
<script>
  function observeResize(commentBlock) {
    const doc = commentBlock.contentDocument.documentElement
    new ResizeObserver(() => {
      commentBlock.style.height = doc.clientHeight + 'px';
    }).observe(doc);
  }
</script>
```

Then a comment system is finished!
User can simply use anchor element with `mailto` URI to send comment:

![comment system](http://topo.tw/photos/2024-12/comments.png)


## For tilde.club

This script could be used by non-admin Unix user, like member of [tilde.club].

For [tilde.club] user, you can easily add it into `.forward` by one-line command:

```bash
echo '"|/home/pham/helper/bin/mail/comment.sh --output_dir /home/<USER>/public_html"' >>~/.forward
```

And then add comment block into web page (`index.html` in this case), please replace recipient of `mailto` with your username:

```bash
...
<div style="border-radius: 6px; background: lightyellow">
  <a style="display: inline-block; margin: 0.5em 0.5em 0 0; float: right" href="mailto:<USER>@tilde.club?subject=Comment on page: /index.html">[Comment on this page]</a>
  <object type="text/html" data="/index.comment.html" onload="observeResize(this)" style="width: 100%;"></object>
  <script>
    function observeResize(commentBlock) {
      const doc = commentBlock.contentDocument.documentElement
      new ResizeObserver(() => {
        commentBlock.style.height = doc.clientHeight + 'px';
      }).observe(doc);
    }
  </script>
</div>
...
```

**That's all you need to do!**


## Improvement


Still, this script only implements very basic features. It could be better with:

1. Build new HTML file from `IMAP` or `Maildir`, because they are stored as mail!
   (In current stage, you will lose all comments if output file is removed)
2. A proper way to show name of commenter. For privacy, this script doesn't expose mail address of sender.
3. Works as CGI to get comment from a web form,
   I don't think every user can/want/expect launch mail client from browser.
   Send a confirmation mail for their comments might be a valid solution.
4. Prevent texts in `<pre>` element fails on checking pattern

I am using this script on my own [site] or tilde.club [page].
Will make changes by my need. You can always get latest version by the following URL:

- https://tilde.club/~pham/comment.sh
- https://git.topo.tw/helper/file/bin/mail/comment.sh.html

Or by `git archive` from my git server:

``` bash
git archive --remote git://git.topo.tw/helper HEAD bin/mail/comment.sh | \
tar --strip=2 -xf -
```

Since this page is working with it, you are welcome to leave comment with any feedback or suggestion!
(Click *Comment on this page* at bottom-right corner)


[^1]: For example, OSM Mailing List: https://lists.openstreetmap.org/listinfo
[^2]: Beware I use term "Message" here.
      For my point of view, a mail is a **well-organized** message.
      Mail uses header to store metadata like information of sender or subject.
      If a MDA is delivering data to a specific destination, then it doesn't matter data is a mail or just a plain-text message.
[^3]: For `Sendmail(1)` in [OpenSMTPD], you should not surround the whole command with double quotes.
      But for [Postfix], double quote is necessary.

[ads]: https://news.ycombinator.com/item?id=13636741
[CGI]: https://tilde.club/wiki/cgi.html
[tilde.club]: https://tilde.club
[alias(5)]: https://man.archlinux.org/man/aliases.5.en
[forward(5)]: https://man.archlinux.org/man/forward.5.en
[unix use this file]: https://www.ibm.com/docs/en/aix/7.3?topic=files-forward-file
[yasuoka-calendar]: https://yasuoka.net/~yasuoka/calendar.html
[sendmail]: https://www.ibm.com/docs/zh-tw/aix/7.3?topic=s-sendmail-command
[mailing list]: https://en.wikipedia.org/wiki/Mailing_list
[RFC 1341]: https://www.jianshu.com/p/f50005a2410c
[OpenSMTPD]: https://www.opensmtpd.org/
[site]: https://topo.tw
[page]: https://tilde.club/~pham/
[FOSS]: abbr:"Free Open Source Software"
[MDA]: abbr:Message Delivery Agent
