This guide shows how to configure search engine indexing for your Sabo app: robots rules, a dynamic sitemap, site-wide metadata (Open Graph/Twitter), and JSON‑LD structured data.
What’s included
- robots:
src/app/robots.ts - sitemap:
src/app/sitemap.ts - Site-wide metadata (Open Graph, Twitter, robots):
src/app/layout.tsx - JSON‑LD (structured data, Organization):
src/app/layout.tsx
Key concepts at a glance
- robots.txt: A plain-text file that tells crawlers which paths they may crawl or must avoid, and where your sitemap lives.
- sitemap.xml: A machine-readable list of your site’s URLs (and optional metadata like last modified). Helps crawlers discover content faster.
- Site-wide metadata (Open Graph, Twitter, robots): Meta tags that control link previews (title, description, image) and crawling behavior.
- JSON‑LD (structured data): Embedded JSON that describes your pages to search engines (e.g., Organization, BlogPosting) to enable rich results.
robots.txt
File:src/app/robots.ts
src/app/robots.ts
1
Set your domain
Change the
sitemap URL to your production domain.Visit
/robots.txt and confirm the sitemap points to https://yourdomain.com/sitemap.xml.2
Control indexing
Add items to
disallow to prevent indexing of private/auth pages. For preview deploys, disallow everything or use a noindex robots policy via site metadata (see below).sitemap.xml
File:src/app/sitemap.ts
src/app/sitemap.ts
1
Set baseUrl
Replace
https://demo.getsabo.com with your production domain (must match robots.ts).2
Add dynamic routes
The example shows blog/changelog from MDX. Add other collections (products, docs, etc.) following the same pattern.
3
Test the output
Visit
/sitemap.xml and verify all routes appear with correct timestamps.The file is
sitemap.ts but Next.js serves it as /sitemap.xml automatically. Use as const for TypeScript strict mode.Site-wide metadata (Open Graph, Twitter, robots)
File:src/app/layout.tsx
src/app/layout.tsx
1
Set metadataBase
Update
metadataBase to your production domain. This becomes the base for all relative URLs in meta tags.2
Customize all fields
Update
title, description, keywords, and social tags (openGraph, twitter) to match your brand.3
Add OG images
Create 1200×630 images in
public/og/. Keep titles/descriptions consistent across openGraph and twitter.Route-level metadata
For specific pages, you can export page-level metadata (orgenerateMetadata) to override the defaults.
JSON‑LD (Structured data)
File:src/app/layout.tsx (Organization). You can add per‑page JSON‑LD (e.g., BlogPosting) in page files.
src/app/layout.tsx
1
Customize fields
Update
name, url, logo, and description. Add social profile URLs to sameAs array.2
Add to layout
Place the script in
<head> of your root layout. The .replace(/</g, "\\u003c") prevents script injection.Verification and testing
1
Manual checks
- Visit
/robots.txtand/sitemap.xml - View page source for meta tags and JSON‑LD
2
Tools
- Google Rich Results Test (structured data)
- OpenGraph Preview tools (OG tags)
- Twitter Card Validator
- Search Console/Bing Webmaster for submission and coverage
3
Common issues
- Wrong domain in
metadataBaseor sitemap - Preview builds accidentally indexed (set
noindex) - OG images missing or wrong path
Deployed site exposes correct robots and sitemap, has consistent OG/Twitter meta, and validates in rich results tests.