Giteabot 7eaf7907d7 Fix HEAD method for robots.txt (#30603) (#30605)
Backport #30603 by @wxiaoguang

Fix #30601


```
~$ curl --head localhost:3000/robots.txt
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=0, private, must-revalidate
Content-Length: 5
Content-Type: text/plain; charset=utf-8
Last-Modified: Wed, 19 Jul 2023 04:56:12 GMT
X-Gitea-Debug: RUN_MODE=dev
Date: Fri, 19 Apr 2024 12:59:44 GMT
```

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-04-19 22:32:49 +02:00
..
2024-03-21 12:02:34 +00:00