Software Engineer Problem Solver

Leo Reading

One Moment, Please...

Find My Articles

Share This Page

B l o g

Blog

Leo Reading

Software Engineer and Problem Solver From the USA

Download Resume

SEO Displaying posts tagged with SEO

Programming

Dynamic Robots.Txt Generator in MVC5 Application

A robots.txt file is important because it tells search engine crawlers what they should and should not crawl.  Unfortunately, they don't always listen, but we should take advantage of the ones that do!  In today's world, websites aren't simply a collection of static pages:  There is ever-changing and new content being posted, and what's there one day may well be gone the next.   Before I begin, I should mention that a lot of this was inspired by Ben Foster's Fabrik.Common library, especially the SiteMap generator.  I took a lot of the same principles and applied them to a slightly different use case, and also abstracted most of it to a class library to avoid copying/pasting every time I need to publish a website.   Robots Txt Generator The robots txt generator was made into a class library so it can be reused across many applications.  It has ...

Continue Reading


One Moment, Please

Close