Drop /robots.txt
Since /audio is now hosted on a separate domain, we don't need robots.txt anymore
This commit is contained in:
parent
1b78d3c2c9
commit
26a989f6b7
3 changed files with 5 additions and 8 deletions
6
Rules
6
Rules
|
@ -46,12 +46,6 @@ compile "/sitemap.xml.erb" do
|
|||
write("/sitemap.xml")
|
||||
end
|
||||
|
||||
##
|
||||
# /robots.txt
|
||||
compile "/robots.txt" do
|
||||
write("/robots.txt")
|
||||
end
|
||||
|
||||
##
|
||||
# Rules
|
||||
passthrough "/json/durations/*.json"
|
||||
|
|
|
@ -2,6 +2,11 @@
|
|||
|
||||
** vNEXT
|
||||
|
||||
**** Remove ~/robots.txt~
|
||||
We don't need this file anymore. It was used to block crawlers
|
||||
indexing audio content at /audio but audio content is now served
|
||||
on a separate domain
|
||||
|
||||
**** Add file mode option to ~nanoc.yaml~
|
||||
The file mode option can be set on the server's unix socket.
|
||||
Default: ~ug=rw,o=~
|
||||
|
|
|
@ -1,2 +0,0 @@
|
|||
user-agent: *
|
||||
disallow: /audio/
|
Loading…
Reference in a new issue