Robots and Sitemap
In this lesson, you will add even more for SEO. Your application deserves robots.txt and proper sitemap.
To make your application more accessible for search engines you should provide them with the robots.txt and sitemap.xml files
Start by informing robots what they should and shouldn't traverse on your website. This helps avoid unnecessary traffic from them. Create a new file, src/robots.txt, and feed it with data:
This configuration will prevent search engine robots from visiting paths that they have no need to traverse:
/favorites is only available to authenticated users.
/login doesn't contain any content that is critical from the SEO perspective.
/assets is where static files reside (this doesn't mean that Google won't index images).
/api is where the backend API resides.
The last line is to inform robots where they should look for the sitemap of your application.