Last night i was looking for a method to remove comments from a robots.txt file containing contents of form below `# robots.txt user-agent Disallow:` Or a more practical format obtained by splitting the robots.txt file line by line. ["# robots.txt", "user-agent", "Disallow:"] every element of the array is a string representing a single line of text from a robots.txt plain text file. Regular expression I used regular expression to strip out the unwanted text until the end of the line however it's not limited to robots.txt it works perfectly with bash.sh files as well. Example var string = "Disallow / #deny bots" string.replace(/#.*$/, "") // returns Disallow / It simply strip's off all characters after the hash character until the end of the line.
Comments
Post a Comment