Add backend-generated sitemap with website manifest sync and nginx SEO proxy support.

Stores synced route config from /meshkee/sitemap-config.json, serves sitemap.xml and robots.txt per tenant, and invalidates cache on CMS writes.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Alireza Hassani
2026-08-22 15:15:32 +03:30
co-authored by Cursor
parent c7d5204b49
commit 2e8d7c1cfd
27 changed files with 1167 additions and 3 deletions
+3
View File
@@ -262,6 +262,9 @@ All routes are prefixed with `/api/v1`.
| POST | `/auth/verify-otp` | Verify OTP (marks cell verified; no tokens) |
| POST | `/auth/handoff/consume` | One-time SSO ticket → tokens (customer → business dashboard) |
| GET | `/tenants/:host` | Resolve business from domain (`specialProductsSource`, `enabledModules`, `homeCharts`, `ePayment`) |
| GET | `/tenants/:host/sitemap.xml` | XML sitemap (static pages from synced manifest + published products/blogs/portfolios) |
| GET | `/tenants/:host/robots.txt` | robots.txt pointing to apex `/sitemap.xml` |
| POST | `/businesses/:id/website/sitemap/sync` | Import manifest from `GET https://{domain}/meshkee/sitemap-config.json` |
| GET | `/tenants/:host/store-specials` | Active store specials (`source` is `product` or `store_item`) |
| GET | `/tenants/:host/website/category-groups` | Homepage category rows |
| GET | `/tenants/:host/website/brand-groups` | Homepage brand rows |
+30
View File
@@ -85,6 +85,36 @@ Use `slot.key` in the placeholder. Match `pageKey` to the website page. For a si
`kind` is `single` (one image) or `list` (duplicatable). For a fixed row, set `itemCount` (e.g. `2`). Leave `itemCount` null for an unbounded slider. `aspectRatio` must look like `16:9`. Do not invent CMS/upload APIs.
### Sitemap (SEO)
Meshkee generates `https://<WEBSITE_DOMAIN>/sitemap.xml` on the API from published CMS content plus a manifest your site publishes. Nginx on the storefront proxies `/sitemap.xml` and `/robots.txt` to the API — do not hardcode a static sitemap file in the repo unless you know nginx is not proxying yet.
**Publish route manifest** so the business dashboard can sync static pages and URL templates:
`GET https://<WEBSITE_DOMAIN>/meshkee/sitemap-config.json`
```json
{
"baseUrl": "https://sanihome.ir",
"staticPages": [
{ "path": "/", "changefreq": "daily", "priority": 1.0 },
{ "path": "/about", "priority": 0.6 },
{ "path": "/contact", "priority": 0.5 },
{ "path": "/blog", "priority": 0.7 }
],
"templates": {
"product": "/products/{slug}",
"blog": "/blog/{slug}",
"portfolio": "/portfolios/{slug}"
}
}
```
- `path` must start with `/`. Do not include checkout, login, or customer-dashboard URLs.
- Each `templates` value must include `{slug}` and match how this site routes detail pages.
- After deploy, the business owner syncs from **Website → Settings** in the dashboard.
- Dynamic URLs (products, blogs, portfolios) come from the CMS; only paths/templates are declared here.
If OpenAPI and this brief conflict, **OpenAPI wins**.
---
+78
View File
@@ -24,6 +24,9 @@
{
"name": "Tenant"
},
{
"name": "SEO"
},
{
"name": "Homepage"
},
@@ -186,6 +189,81 @@
}
}
},
"/tenants/{domain}/sitemap.xml": {
"get": {
"tags": [
"SEO"
],
"summary": "XML sitemap for search engines",
"description": "Published products, blogs, and portfolios for this tenant (respects enabled modules), plus static pages and URL templates from the synced website manifest (`GET https://{domain}/meshkee/sitemap-config.json`). Default paths when not synced: `/`, `/products/{slug}`, `/blog/{slug}`, `/portfolios/{slug}`. Proxied from `https://{domain}/sitemap.xml` on the storefront. Cached in Redis; refreshed after CMS writes and manifest sync.",
"parameters": [
{
"$ref": "#/components/parameters/domain"
}
],
"responses": {
"200": {
"description": "Sitemap XML (urlset)",
"content": {
"application/xml": {
"schema": {
"type": "string"
}
}
},
"headers": {
"Cache-Control": {
"schema": {
"type": "string",
"example": "public, max-age=900"
}
}
}
},
"404": {
"description": "Unknown domain"
}
}
}
},
"/tenants/{domain}/robots.txt": {
"get": {
"tags": [
"SEO"
],
"summary": "robots.txt with sitemap reference",
"description": "Plain-text robots file pointing crawlers to `https://{domain}/sitemap.xml`. Proxied from `https://{domain}/robots.txt` on the storefront.",
"parameters": [
{
"$ref": "#/components/parameters/domain"
}
],
"responses": {
"200": {
"description": "robots.txt",
"content": {
"text/plain": {
"schema": {
"type": "string",
"example": "User-agent: *\\nAllow: /\\n\\nSitemap: https://example.com/sitemap.xml\\n"
}
}
},
"headers": {
"Cache-Control": {
"schema": {
"type": "string",
"example": "public, max-age=900"
}
}
}
},
"404": {
"description": "Unknown domain"
}
}
}
},
"/tenants/{domain}/website/business-info": {
"get": {
"tags": [
+3
View File
@@ -9,6 +9,9 @@
# POST /ssl { host, slug? } — certbot for apex + www (nginx must exist)
# GET /health
#
# Scripts on the VM (same folder):
# patch-nginx-seo.sh <host> — add /sitemap.xml + /robots.txt API proxies to an existing nginx site
#
# Env (.env): PORT, DEPLOY_TOKEN, ALLOWED_SLUGS
#
# Note: provision.sh intentionally skips certbot. SSL is issued via POST /ssl so
+61
View File
@@ -0,0 +1,61 @@
#!/usr/bin/env bash
# Insert Meshkee SEO proxy locations (/sitemap.xml, /robots.txt) into an existing storefront nginx site.
set -euo pipefail
HOST="${1:-}"
API_HOST="${MESHKEE_API_HOST:-api.meshkee.com}"
if [[ -z "$HOST" ]]; then
echo "usage: patch-nginx-seo.sh <host>" >&2
exit 1
fi
NGINX_AVAILABLE="/etc/nginx/sites-available/$HOST"
if [[ ! -f "$NGINX_AVAILABLE" ]]; then
echo "missing nginx site for $HOST" >&2
exit 1
fi
if grep -q 'location = /sitemap.xml' "$NGINX_AVAILABLE"; then
echo "nginx SEO locations already present for $HOST"
exit 0
fi
TMP="$(mktemp)"
SEO_BLOCK="$(cat <<NGINX
location = /sitemap.xml {
proxy_pass https://${API_HOST}/api/v1/tenants/${HOST}/sitemap.xml;
proxy_set_header Host ${API_HOST};
proxy_ssl_server_name on;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto \$scheme;
add_header Cache-Control "public, max-age=900";
}
location = /robots.txt {
proxy_pass https://${API_HOST}/api/v1/tenants/${HOST}/robots.txt;
proxy_set_header Host ${API_HOST};
proxy_ssl_server_name on;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto \$scheme;
add_header Cache-Control "public, max-age=900";
}
NGINX
)"
awk -v block="$SEO_BLOCK" '
/location \/ \{/ && !done {
print block
done = 1
}
{ print }
' "$NGINX_AVAILABLE" >"$TMP"
mv "$TMP" "$NGINX_AVAILABLE"
nginx -t
systemctl reload nginx
echo "patched nginx SEO locations for $HOST"
+24
View File
@@ -122,6 +122,26 @@ server {
listen [::]:80;
server_name ${HOST} www.${HOST};
location = /sitemap.xml {
proxy_pass https://api.meshkee.com/api/v1/tenants/${HOST}/sitemap.xml;
proxy_set_header Host api.meshkee.com;
proxy_ssl_server_name on;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto \$scheme;
add_header Cache-Control "public, max-age=900";
}
location = /robots.txt {
proxy_pass https://api.meshkee.com/api/v1/tenants/${HOST}/robots.txt;
proxy_set_header Host api.meshkee.com;
proxy_ssl_server_name on;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto \$scheme;
add_header Cache-Control "public, max-age=900";
}
location / {
proxy_pass http://127.0.0.1:${PORT};
proxy_http_version 1.1;
@@ -141,6 +161,10 @@ NGINX
echo "nginx site created for $HOST → :$PORT"
else
echo "nginx site already exists: $NGINX_AVAILABLE"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [[ -x "$SCRIPT_DIR/patch-nginx-seo.sh" ]]; then
"$SCRIPT_DIR/patch-nginx-seo.sh" "$HOST" || true
fi
fi
# SSL is issued separately via ssl.sh / Super Admin "Issue SSL" — do not run
+2
View File
@@ -40,6 +40,7 @@ import { InvoicesModule } from './invoices/invoices.module';
import { LegacyMysqlModule } from './legacy-mysql/legacy-mysql.module';
import { PublicSmsModule } from './public-sms/public-sms.module';
import { PaymentsModule } from './payments/payments.module';
import { SitemapModule } from './sitemap/sitemap.module';
@Module({
imports: [
@@ -84,6 +85,7 @@ import { PaymentsModule } from './payments/payments.module';
WebsiteDocsModule,
InvoicesModule,
PublicSmsModule,
SitemapModule,
],
})
export class AppModule {}
+2 -1
View File
@@ -1,12 +1,13 @@
import { Module } from '@nestjs/common';
import { AuthModule } from '../auth/auth.module';
import { BusinessSettingsModule } from '../business-settings/business-settings.module';
import { SitemapModule } from '../sitemap/sitemap.module';
import { TenantModule } from '../tenant/tenant.module';
import { BlogsController, PublicBlogsController } from './blogs.controller';
import { BlogsService } from './blogs.service';
@Module({
imports: [AuthModule, BusinessSettingsModule, TenantModule],
imports: [AuthModule, BusinessSettingsModule, SitemapModule, TenantModule],
controllers: [BlogsController, PublicBlogsController],
providers: [BlogsService],
})
+8
View File
@@ -13,6 +13,7 @@ import { AuthUser } from '../auth/auth.types';
import { PermissionsService } from '../auth/permissions.service';
import { BusinessSettingsService } from '../business-settings/business-settings.service';
import { PrismaService } from '../prisma/prisma.service';
import { SitemapService } from '../sitemap/sitemap.service';
import { TenantService } from '../tenant/tenant.service';
import {
CreateBlogCommentDto,
@@ -58,6 +59,7 @@ export class BlogsService {
private readonly permissions: PermissionsService,
private readonly tenant: TenantService,
private readonly businessSettings: BusinessSettingsService,
private readonly sitemap: SitemapService,
) {}
async list(businessIdRaw: string, query: ListBlogsDto, actor: AuthUser) {
@@ -155,6 +157,8 @@ export class BlogsService {
return blog;
});
await this.sitemap.invalidateForBusiness(businessId);
return {
message: 'Blog post created successfully',
blog: await this.serializeBlog(created),
@@ -275,6 +279,8 @@ export class BlogsService {
return blog;
});
await this.sitemap.invalidateForBusiness(businessId);
return {
message: 'Blog post updated successfully',
blog: await this.serializeBlog(updated),
@@ -312,6 +318,8 @@ export class BlogsService {
this.prisma.blogs.delete({ where: { id: blogId } }),
]);
await this.sitemap.invalidateForBusiness(businessId);
return { message: 'Blog post deleted successfully' };
}
@@ -107,6 +107,41 @@ export type SpecialProductsSource = (typeof SPECIAL_PRODUCTS_SOURCE_IDS)[number]
/** Per-business website CMS settings. */
export type WebsiteSettings = {
specialProductsSource: SpecialProductsSource;
/** Imported from GET https://{domain}/meshkee/sitemap-config.json */
sitemapConfig?: WebsiteSitemapConfig | null;
};
export type SitemapChangefreq =
| 'always'
| 'hourly'
| 'daily'
| 'weekly'
| 'monthly'
| 'yearly'
| 'never';
export type SitemapStaticPageConfig = {
path: string;
changefreq?: SitemapChangefreq;
priority?: number;
};
export type SitemapPathTemplatesConfig = {
product?: string;
blog?: string;
portfolio?: string;
storeItemByProduct?: string;
video?: string;
instruction?: string;
workshop?: string;
userProduct?: string;
};
export type WebsiteSitemapConfig = {
baseUrl?: string | null;
staticPages?: SitemapStaticPageConfig[];
templates?: SitemapPathTemplatesConfig;
syncedAt?: string | null;
};
/** Optional business-dashboard CMS modules. Always-on areas (customers, website, etc.) are not listed. */
@@ -2,6 +2,9 @@ import { Prisma } from '@prisma/client';
import {
normalizeBusinessPrimaryColorId,
} from './business-primary-colors';
import {
normalizeWebsiteSitemapConfig,
} from '../sitemap/sitemap-config.util';
import {
BUSINESS_MODULE_IDS,
BusinessModuleId,
@@ -28,6 +31,7 @@ import {
type SpecialProductsSource,
OrderProcessStep,
StubGatewaySettings,
WebsiteSitemapConfig,
ZarinpalGatewaySettings,
} from './business-settings.types';
import {
@@ -412,6 +416,7 @@ export function normalizeBusinessSettings(raw: unknown): BusinessSettings {
website.specialProductsSource,
enabledModules,
),
sitemapConfig: normalizeWebsiteSitemapConfig(website.sitemapConfig),
},
};
}
@@ -458,6 +463,10 @@ export function mergeBusinessSettings(
specialProductsSource:
patch.website?.specialProductsSource ??
current.website.specialProductsSource,
sitemapConfig:
patch.website?.sitemapConfig !== undefined
? patch.website.sitemapConfig
: current.website.sitemapConfig,
},
};
}
+2 -1
View File
@@ -1,6 +1,7 @@
import { Module } from '@nestjs/common';
import { AuthModule } from '../auth/auth.module';
import { BusinessSettingsModule } from '../business-settings/business-settings.module';
import { SitemapModule } from '../sitemap/sitemap.module';
import { TenantModule } from '../tenant/tenant.module';
import {
PortfoliosController,
@@ -9,7 +10,7 @@ import {
import { PortfoliosService } from './portfolios.service';
@Module({
imports: [AuthModule, BusinessSettingsModule, TenantModule],
imports: [AuthModule, BusinessSettingsModule, SitemapModule, TenantModule],
controllers: [PortfoliosController, PublicPortfoliosController],
providers: [PortfoliosService],
})
+8
View File
@@ -13,6 +13,7 @@ import { AuthUser } from '../auth/auth.types';
import { PermissionsService } from '../auth/permissions.service';
import { BusinessSettingsService } from '../business-settings/business-settings.service';
import { PrismaService } from '../prisma/prisma.service';
import { SitemapService } from '../sitemap/sitemap.service';
import { TenantService } from '../tenant/tenant.service';
import {
CreatePortfolioCommentDto,
@@ -47,6 +48,7 @@ export class PortfoliosService {
private readonly permissions: PermissionsService,
private readonly tenant: TenantService,
private readonly businessSettings: BusinessSettingsService,
private readonly sitemap: SitemapService,
) {}
async list(businessIdRaw: string, query: ListPortfoliosDto, actor: AuthUser) {
@@ -181,6 +183,8 @@ export class PortfoliosService {
return portfolio;
});
await this.sitemap.invalidateForBusiness(businessId);
return {
message: 'Portfolio created successfully',
portfolio: await this.serializePortfolio(created),
@@ -344,6 +348,8 @@ export class PortfoliosService {
return portfolio;
});
await this.sitemap.invalidateForBusiness(businessId);
return {
message: 'Portfolio updated successfully',
portfolio: await this.serializePortfolio(updated),
@@ -392,6 +398,8 @@ export class PortfoliosService {
this.prisma.portfolios.delete({ where: { id: portfolioId } }),
]);
await this.sitemap.invalidateForBusiness(businessId);
return { message: 'Portfolio deleted successfully' };
}
+2 -1
View File
@@ -2,6 +2,7 @@ import { Module } from '@nestjs/common';
import { AuthModule } from '../auth/auth.module';
import { BrandsModule } from '../brands/brands.module';
import { CategoriesModule } from '../categories/categories.module';
import { SitemapModule } from '../sitemap/sitemap.module';
import { TenantModule } from '../tenant/tenant.module';
import { ProductAiService } from './product-ai.service';
import { ProductTechnicalInfoService } from './product-technical-info.service';
@@ -10,7 +11,7 @@ import { ProductsController, PublicProductsController } from './products.control
import { ProductsService } from './products.service';
@Module({
imports: [AuthModule, BrandsModule, CategoriesModule, TenantModule],
imports: [AuthModule, BrandsModule, CategoriesModule, SitemapModule, TenantModule],
controllers: [ProductsController, PublicProductsController],
providers: [
ProductsService,
+8
View File
@@ -9,6 +9,7 @@ import { AuthUser } from '../auth/auth.types';
import { PermissionsService } from '../auth/permissions.service';
import { BrandsService } from '../brands/brands.service';
import { PrismaService } from '../prisma/prisma.service';
import { SitemapService } from '../sitemap/sitemap.service';
import { TenantService } from '../tenant/tenant.service';
import { CreateProductDto, ListProductsDto, ListPublicProductsDto, UpdateProductDto } from './dto/product.dto';
import { ProductVariationValuesService } from './product-variation-values.service';
@@ -52,6 +53,7 @@ export class ProductsService {
private readonly productVariationValues: ProductVariationValuesService,
private readonly brands: BrandsService,
private readonly tenant: TenantService,
private readonly sitemap: SitemapService,
) {}
async list(businessIdRaw: string, query: ListProductsDto, actor: AuthUser) {
@@ -276,6 +278,8 @@ export class ProductsService {
return product;
});
await this.sitemap.invalidateForBusiness(businessId);
return {
message: 'Product created successfully',
product: await this.serializeProduct(created),
@@ -414,6 +418,8 @@ export class ProductsService {
return product;
});
await this.sitemap.invalidateForBusiness(businessId);
return {
message: 'Product updated successfully',
product: await this.serializeProduct(updated),
@@ -451,6 +457,8 @@ export class ProductsService {
this.prisma.product.delete({ where: { id: productId } }),
]);
await this.sitemap.invalidateForBusiness(businessId);
return { message: 'Product deleted successfully' };
}
+22
View File
@@ -0,0 +1,22 @@
import { Controller, Post, Param, UseGuards } from '@nestjs/common';
import { AuthUser } from '../auth/auth.types';
import { CurrentUser } from '../auth/decorators/current-user.decorator';
import { RequireBusinessPermission } from '../auth/decorators/require-business-permission.decorator';
import { BusinessPermissionGuard } from '../auth/guards/business-permission.guard';
import { JwtAuthGuard } from '../auth/guards/jwt-auth.guard';
import { SitemapConfigService } from './sitemap-config.service';
@Controller('businesses/:businessId/website/sitemap')
@UseGuards(JwtAuthGuard, BusinessPermissionGuard)
export class SitemapAdminController {
constructor(private readonly config: SitemapConfigService) {}
@Post('sync')
@RequireBusinessPermission('website.update')
syncFromWebsite(
@Param('businessId') businessId: string,
@CurrentUser() user: AuthUser,
) {
return this.config.syncFromWebsite(businessId, user);
}
}
+151
View File
@@ -0,0 +1,151 @@
import type {
SitemapChangefreq,
SitemapPathTemplatesConfig,
SitemapStaticPageConfig,
WebsiteSitemapConfig,
} from '../business-settings/business-settings.types';
export const WEBSITE_SITEMAP_CONFIG_PATH = '/meshkee/sitemap-config.json';
export const SITEMAP_CHANGEFREQ_VALUES = [
'always',
'hourly',
'daily',
'weekly',
'monthly',
'yearly',
'never',
] as const;
const PATH_PATTERN = /^\/[^\s?#]*$/;
const TEMPLATE_PATTERN = /^\/[^\s?#]*\{slug\}[^\s?#]*$/;
const BASE_URL_PATTERN = /^https:\/\/[a-z0-9.-]+(?::\d+)?\/?$/i;
const TEMPLATE_KEYS = [
'product',
'blog',
'portfolio',
'storeItemByProduct',
'video',
'instruction',
'workshop',
'userProduct',
] as const satisfies readonly (keyof SitemapPathTemplatesConfig)[];
function asRecord(value: unknown): Record<string, unknown> | null {
return value && typeof value === 'object' && !Array.isArray(value)
? (value as Record<string, unknown>)
: null;
}
function readString(value: unknown): string {
return typeof value === 'string' ? value.trim() : '';
}
function readChangefreq(value: unknown): SitemapChangefreq | undefined {
const raw = readString(value);
return (SITEMAP_CHANGEFREQ_VALUES as readonly string[]).includes(raw)
? (raw as SitemapChangefreq)
: undefined;
}
function readPriority(value: unknown): number | undefined {
if (value == null || value === '') return undefined;
const parsed = typeof value === 'number' ? value : Number(value);
if (!Number.isFinite(parsed)) return undefined;
return Math.min(1, Math.max(0, parsed));
}
function readPath(value: unknown): string | null {
const path = readString(value);
if (!path || !PATH_PATTERN.test(path)) return null;
return path;
}
function readTemplate(value: unknown): string | null {
const template = readString(value);
if (!template || !TEMPLATE_PATTERN.test(template)) return null;
return template;
}
function readBaseUrl(value: unknown): string | null {
const url = readString(value).replace(/\/+$/, '');
if (!url || !BASE_URL_PATTERN.test(`${url}/`)) return null;
return url;
}
export function parseWebsiteSitemapConfig(payload: unknown): WebsiteSitemapConfig {
const root = asRecord(payload);
if (!root) {
throw new Error('Sitemap config must be a JSON object');
}
const staticPages: SitemapStaticPageConfig[] = [];
const seenPaths = new Set<string>();
if (Array.isArray(root.staticPages)) {
for (const entry of root.staticPages) {
const row = asRecord(entry);
if (!row) continue;
const path = readPath(row.path);
if (!path || seenPaths.has(path)) continue;
seenPaths.add(path);
staticPages.push({
path,
...(readChangefreq(row.changefreq)
? { changefreq: readChangefreq(row.changefreq) }
: {}),
...(readPriority(row.priority) != null
? { priority: readPriority(row.priority) }
: {}),
});
}
}
const templatesRaw = asRecord(root.templates);
const templates: SitemapPathTemplatesConfig = {};
if (templatesRaw) {
for (const key of TEMPLATE_KEYS) {
const template = readTemplate(templatesRaw[key]);
if (template) {
templates[key] = template;
}
}
}
const baseUrl = readBaseUrl(root.baseUrl);
if (staticPages.length === 0 && !baseUrl && Object.keys(templates).length === 0) {
throw new Error(
'Sitemap config must include staticPages, templates, or baseUrl',
);
}
return {
...(baseUrl ? { baseUrl } : {}),
...(staticPages.length > 0 ? { staticPages } : {}),
...(Object.keys(templates).length > 0 ? { templates } : {}),
syncedAt: new Date().toISOString(),
};
}
export function websiteSitemapConfigUrls(hosts: string[]): string[] {
const uniqueHosts = [
...new Set(hosts.map((host) => host.toLowerCase().trim()).filter(Boolean)),
];
const urls: string[] = [];
for (const host of uniqueHosts) {
const bare = host.replace(/^www\./, '');
const variants = [bare, `www.${bare}`];
for (const candidate of variants) {
urls.push(`https://${candidate}${WEBSITE_SITEMAP_CONFIG_PATH}`);
urls.push(`http://${candidate}${WEBSITE_SITEMAP_CONFIG_PATH}`);
}
}
return [...new Set(urls)];
}
+145
View File
@@ -0,0 +1,145 @@
import {
BadRequestException,
ForbiddenException,
Injectable,
NotFoundException,
} from '@nestjs/common';
import { AuthUser } from '../auth/auth.types';
import { PermissionsService } from '../auth/permissions.service';
import {
mergeBusinessSettings,
normalizeBusinessSettings,
toPrismaJson,
} from '../business-settings/business-settings.util';
import { PrismaService } from '../prisma/prisma.service';
import {
parseWebsiteSitemapConfig,
WEBSITE_SITEMAP_CONFIG_PATH,
websiteSitemapConfigUrls,
} from './sitemap-config.catalog';
import { SitemapService } from './sitemap.service';
@Injectable()
export class SitemapConfigService {
constructor(
private readonly prisma: PrismaService,
private readonly permissions: PermissionsService,
private readonly sitemap: SitemapService,
) {}
async syncFromWebsite(businessIdRaw: string, actor: AuthUser) {
const businessId = BigInt(businessIdRaw);
await this.assertPermission(businessId, actor.id, 'website.update');
const business = await this.prisma.business.findUnique({
where: { id: businessId },
select: { id: true, settings: true },
});
if (!business) {
throw new NotFoundException('Business not found');
}
const hosts = await this.listWebsiteHosts(businessId);
if (hosts.length === 0) {
throw new BadRequestException(
'This business has no website domain to read sitemap config from',
);
}
const parsed = await this.fetchWebsiteConfig(hosts);
const current = normalizeBusinessSettings(business.settings);
const next = mergeBusinessSettings(current, {
website: {
...current.website,
sitemapConfig: parsed,
},
});
await this.prisma.business.update({
where: { id: businessId },
data: { settings: toPrismaJson(next) },
});
await this.sitemap.invalidateForBusiness(businessId);
return {
message: 'Sitemap config refreshed from the website',
sitemapConfig: next.website.sitemapConfig,
};
}
private async listWebsiteHosts(businessId: bigint): Promise<string[]> {
const domains = await this.prisma.domain.findMany({
where: { businessId, isActive: true },
orderBy: [{ isPrimary: 'desc' }, { host: 'asc' }],
select: { host: true },
});
return domains
.map((domain) => domain.host.toLowerCase().trim())
.filter(
(host) =>
host &&
!host.startsWith('business.') &&
!host.startsWith('customer.'),
);
}
private async fetchWebsiteConfig(hosts: string[]) {
const urls = websiteSitemapConfigUrls(hosts);
let lastError = 'Website sitemap config was not found';
for (const url of urls) {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 8000);
try {
const response = await fetch(url, {
method: 'GET',
redirect: 'follow',
signal: controller.signal,
headers: {
Accept: 'application/json',
'User-Agent': 'Meshkee-CMS/1.0',
},
});
if (!response.ok) {
lastError = `Website returned ${response.status} for ${WEBSITE_SITEMAP_CONFIG_PATH}`;
continue;
}
const payload: unknown = await response.json();
return parseWebsiteSitemapConfig(payload);
} catch (error) {
if (error instanceof Error && error.message.startsWith('Sitemap config')) {
lastError = error.message;
continue;
}
lastError = `Could not read ${WEBSITE_SITEMAP_CONFIG_PATH} from the website`;
} finally {
clearTimeout(timer);
}
}
throw new BadRequestException(
`${lastError}. The site must publish GET ${WEBSITE_SITEMAP_CONFIG_PATH} on its main domain.`,
);
}
private async assertPermission(
businessId: bigint,
userId: bigint,
permission: string,
) {
const allowed = await this.permissions.hasBusinessPermission(
userId,
businessId,
permission,
);
if (!allowed) {
throw new ForbiddenException(
`Missing permission: ${permission} for this business`,
);
}
}
}
+17
View File
@@ -0,0 +1,17 @@
export type {
SitemapChangefreq,
SitemapPathTemplatesConfig,
SitemapStaticPageConfig,
WebsiteSitemapConfig,
} from '../business-settings/business-settings.types';
export type ResolvedSitemapConfig = {
baseUrl: string;
staticPages: import('../business-settings/business-settings.types').SitemapStaticPageConfig[];
templates: Required<
Pick<
import('../business-settings/business-settings.types').SitemapPathTemplatesConfig,
'product' | 'blog' | 'portfolio'
>
>;
};
+103
View File
@@ -0,0 +1,103 @@
import {
DEFAULT_SITEMAP_PATH_TEMPLATES,
DEFAULT_SITEMAP_STATIC_PAGES,
} from './sitemap.constants';
import type {
ResolvedSitemapConfig,
SitemapStaticPageConfig,
WebsiteSitemapConfig,
} from './sitemap-config.types';
function normalizeStoredStaticPages(
value: unknown,
): SitemapStaticPageConfig[] | null {
if (!Array.isArray(value)) return null;
const pages: SitemapStaticPageConfig[] = [];
const seen = new Set<string>();
for (const entry of value) {
if (!entry || typeof entry !== 'object') continue;
const row = entry as Record<string, unknown>;
const path = typeof row.path === 'string' ? row.path.trim() : '';
if (!path.startsWith('/') || seen.has(path)) continue;
seen.add(path);
pages.push({
path,
...(typeof row.changefreq === 'string'
? { changefreq: row.changefreq as SitemapStaticPageConfig['changefreq'] }
: {}),
...(typeof row.priority === 'number' ? { priority: row.priority } : {}),
});
}
return pages.length > 0 ? pages : null;
}
function normalizeStoredTemplates(
value: unknown,
): WebsiteSitemapConfig['templates'] | null {
if (!value || typeof value !== 'object') return null;
const raw = value as Record<string, unknown>;
const templates: NonNullable<WebsiteSitemapConfig['templates']> = {};
for (const [key, template] of Object.entries(raw)) {
if (typeof template === 'string' && template.includes('{slug}')) {
templates[key as keyof typeof templates] = template.trim();
}
}
return Object.keys(templates).length > 0 ? templates : null;
}
export function normalizeWebsiteSitemapConfig(
raw: unknown,
): WebsiteSitemapConfig | null {
if (!raw || typeof raw !== 'object') return null;
const source = raw as Record<string, unknown>;
const staticPages = normalizeStoredStaticPages(source.staticPages);
const templates = normalizeStoredTemplates(source.templates);
const baseUrl =
typeof source.baseUrl === 'string' && source.baseUrl.trim()
? source.baseUrl.trim().replace(/\/+$/, '')
: null;
const syncedAt =
typeof source.syncedAt === 'string' && source.syncedAt.trim()
? source.syncedAt.trim()
: null;
if (!staticPages && !templates && !baseUrl) return null;
return {
...(baseUrl ? { baseUrl } : {}),
...(staticPages ? { staticPages } : {}),
...(templates ? { templates } : {}),
...(syncedAt ? { syncedAt } : {}),
};
}
export function resolveSitemapConfig(
stored: WebsiteSitemapConfig | null | undefined,
apexHost: string,
): ResolvedSitemapConfig {
const fallbackBase = `https://${apexHost.toLowerCase().trim()}`;
const baseUrl = stored?.baseUrl?.replace(/\/+$/, '') || fallbackBase;
const staticPages =
stored?.staticPages && stored.staticPages.length > 0
? stored.staticPages
: [...DEFAULT_SITEMAP_STATIC_PAGES];
return {
baseUrl,
staticPages,
templates: {
product:
stored?.templates?.product ?? DEFAULT_SITEMAP_PATH_TEMPLATES.product,
blog: stored?.templates?.blog ?? DEFAULT_SITEMAP_PATH_TEMPLATES.blog,
portfolio:
stored?.templates?.portfolio ??
DEFAULT_SITEMAP_PATH_TEMPLATES.portfolio,
},
};
}
+63
View File
@@ -0,0 +1,63 @@
import { SITEMAP_URLSET_NS } from './sitemap.constants';
export type SitemapUrlEntry = {
loc: string;
lastmod?: Date | null;
changefreq?: 'always' | 'hourly' | 'daily' | 'weekly' | 'monthly' | 'yearly' | 'never';
priority?: number;
};
function escapeXml(value: string): string {
return value
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&apos;');
}
function formatLastmod(date: Date): string {
return date.toISOString();
}
function formatPriority(value: number): string {
const clamped = Math.min(1, Math.max(0, value));
return clamped.toFixed(1);
}
export function buildSitemapXml(entries: SitemapUrlEntry[]): string {
const urls = entries
.map((entry) => {
const parts = [` <loc>${escapeXml(entry.loc)}</loc>`];
if (entry.lastmod) {
parts.push(` <lastmod>${formatLastmod(entry.lastmod)}</lastmod>`);
}
if (entry.changefreq) {
parts.push(` <changefreq>${entry.changefreq}</changefreq>`);
}
if (entry.priority != null) {
parts.push(` <priority>${formatPriority(entry.priority)}</priority>`);
}
return ` <url>\n${parts.join('\n')}\n </url>`;
})
.join('\n');
return [
'<?xml version="1.0" encoding="UTF-8"?>',
`<urlset xmlns="${SITEMAP_URLSET_NS}">`,
urls,
'</urlset>',
'',
].join('\n');
}
export function applyPathTemplate(template: string, slug: string): string {
return template.replace('{slug}', encodeURIComponent(slug));
}
export function buildAbsoluteUrl(baseUrl: string, path: string): string {
const normalizedBase = baseUrl.replace(/\/+$/, '');
if (path === '/') return `${normalizedBase}/`;
const normalizedPath = path.startsWith('/') ? path : `/${path}`;
return `${normalizedBase}${normalizedPath}`;
}
+17
View File
@@ -0,0 +1,17 @@
/** Safety-net TTL when invalidation is missed; normal freshness is event-driven. */
export const SITEMAP_CACHE_TTL_SECONDS = 3600;
export const SITEMAP_URLSET_NS = 'http://www.sitemaps.org/schemas/sitemap/0.9';
/** Default storefront path templates (overridable via /meshkee/sitemap-config.json). */
export const DEFAULT_SITEMAP_PATH_TEMPLATES = {
product: '/products/{slug}',
blog: '/blog/{slug}',
portfolio: '/portfolios/{slug}',
} as const;
export const WEBSITE_SITEMAP_CONFIG_PATH = '/meshkee/sitemap-config.json';
export const DEFAULT_SITEMAP_STATIC_PAGES = [
{ path: '/', changefreq: 'daily' as const, priority: 1.0 },
] as const;
+24
View File
@@ -0,0 +1,24 @@
import { Controller, Get, Header, Param, Res } from '@nestjs/common';
import type { Response } from 'express';
import { SitemapService } from './sitemap.service';
@Controller('tenants/:host')
export class SitemapController {
constructor(private readonly sitemap: SitemapService) {}
@Get('sitemap.xml')
@Header('Cache-Control', 'public, max-age=900')
async getSitemap(@Param('host') host: string, @Res() res: Response) {
const xml = await this.sitemap.getXmlForHost(host);
res.setHeader('Content-Type', 'application/xml; charset=utf-8');
res.send(xml);
}
@Get('robots.txt')
@Header('Cache-Control', 'public, max-age=900')
async getRobots(@Param('host') host: string, @Res() res: Response) {
const text = await this.sitemap.getRobotsForHost(host);
res.setHeader('Content-Type', 'text/plain; charset=utf-8');
res.send(text);
}
}
+15
View File
@@ -0,0 +1,15 @@
import { Module } from '@nestjs/common';
import { AuthModule } from '../auth/auth.module';
import { TenantModule } from '../tenant/tenant.module';
import { SitemapAdminController } from './sitemap-admin.controller';
import { SitemapConfigService } from './sitemap-config.service';
import { SitemapController } from './sitemap.controller';
import { SitemapService } from './sitemap.service';
@Module({
imports: [AuthModule, TenantModule],
controllers: [SitemapController, SitemapAdminController],
providers: [SitemapService, SitemapConfigService],
exports: [SitemapService],
})
export class SitemapModule {}
+227
View File
@@ -0,0 +1,227 @@
import { Injectable, Logger } from '@nestjs/common';
import { ContentStatus } from '@prisma/client';
import { normalizeBusinessSettings } from '../business-settings/business-settings.util';
import type { BusinessModuleId } from '../business-settings/business-settings.types';
import { PrismaService } from '../prisma/prisma.service';
import { RedisService } from '../redis/redis.service';
import { TenantService } from '../tenant/tenant.service';
import { resolveSitemapConfig } from './sitemap-config.util';
import { SITEMAP_CACHE_TTL_SECONDS } from './sitemap.constants';
import {
applyPathTemplate,
buildAbsoluteUrl,
buildSitemapXml,
type SitemapUrlEntry,
} from './sitemap-xml.util';
type PublishedSlugRow = {
slug: string;
updatedAt: Date;
};
@Injectable()
export class SitemapService {
private readonly logger = new Logger(SitemapService.name);
constructor(
private readonly prisma: PrismaService,
private readonly redis: RedisService,
private readonly tenant: TenantService,
) {}
private cacheKey(businessId: bigint): string {
return `sitemap:${businessId.toString()}`;
}
async invalidateForBusiness(businessId: bigint | string): Promise<void> {
const id = typeof businessId === 'bigint' ? businessId : BigInt(businessId);
try {
await this.redis.client.del(this.cacheKey(id));
} catch (err) {
this.logger.warn(
`Failed to invalidate sitemap cache for business ${id.toString()}: ${String(err)}`,
);
}
}
async getXmlForHost(host: string): Promise<string> {
const business = await this.tenant.resolveBusinessByDomain(host);
const cacheKey = this.cacheKey(business.id);
try {
const cached = await this.redis.client.get(cacheKey);
if (cached) return cached;
} catch (err) {
this.logger.warn(`Sitemap cache read failed: ${String(err)}`);
}
const apexHost = await this.resolveApexHost(business.id, host);
const xml = await this.buildXml(business.id, apexHost);
try {
await this.redis.client.set(cacheKey, xml, 'EX', SITEMAP_CACHE_TTL_SECONDS);
} catch (err) {
this.logger.warn(`Sitemap cache write failed: ${String(err)}`);
}
return xml;
}
async getRobotsForHost(host: string): Promise<string> {
const business = await this.tenant.resolveBusinessByDomain(host);
const apexHost = await this.resolveApexHost(business.id, host);
const stored = await this.prisma.business.findUnique({
where: { id: business.id },
select: { settings: true },
});
const settings = normalizeBusinessSettings(stored?.settings);
const config = resolveSitemapConfig(
settings.website.sitemapConfig,
apexHost,
);
return ['User-agent: *', 'Allow: /', '', `Sitemap: ${config.baseUrl}/sitemap.xml`, ''].join(
'\n',
);
}
private async resolveApexHost(businessId: bigint, fallbackHost: string): Promise<string> {
const primary = await this.prisma.domain.findFirst({
where: { businessId, isPrimary: true },
select: { host: true },
});
return primary?.host ?? fallbackHost.toLowerCase().trim();
}
private isModuleEnabled(
enabledModules: readonly BusinessModuleId[],
moduleId: BusinessModuleId,
): boolean {
return enabledModules.includes(moduleId);
}
private async buildXml(businessId: bigint, apexHost: string): Promise<string> {
const business = await this.prisma.business.findUnique({
where: { id: businessId },
select: { settings: true },
});
const settings = normalizeBusinessSettings(business?.settings);
const enabledModules = settings.modules.enabled;
const config = resolveSitemapConfig(settings.website.sitemapConfig, apexHost);
const entries: SitemapUrlEntry[] = config.staticPages.map((page) => ({
loc: buildAbsoluteUrl(config.baseUrl, page.path),
...(page.changefreq ? { changefreq: page.changefreq } : {}),
...(page.priority != null ? { priority: page.priority } : {}),
}));
if (this.isModuleEnabled(enabledModules, 'products')) {
entries.push(
...(await this.loadPublishedProducts(businessId, config.baseUrl, config.templates.product)),
);
}
if (this.isModuleEnabled(enabledModules, 'blog')) {
entries.push(
...(await this.loadPublishedBlogs(businessId, config.baseUrl, config.templates.blog)),
);
}
if (this.isModuleEnabled(enabledModules, 'portfolio')) {
entries.push(
...(await this.loadPublishedPortfolios(
businessId,
config.baseUrl,
config.templates.portfolio,
)),
);
}
return buildSitemapXml(entries);
}
private async loadPublishedProducts(
businessId: bigint,
baseUrl: string,
template: string,
): Promise<SitemapUrlEntry[]> {
const rows = await this.prisma.product.findMany({
where: {
businessId,
status: ContentStatus.published,
},
select: {
slug: true,
updatedAt: true,
},
orderBy: [{ updatedAt: 'desc' }, { id: 'desc' }],
});
return rows.map((row) => this.toEntry(baseUrl, template, row));
}
private async loadPublishedBlogs(
businessId: bigint,
baseUrl: string,
template: string,
): Promise<SitemapUrlEntry[]> {
const rows = await this.prisma.blogs.findMany({
where: {
business_id: businessId,
status: ContentStatus.published,
},
select: {
slug: true,
updated_at: true,
},
orderBy: [{ updated_at: 'desc' }, { id: 'desc' }],
});
return rows.map((row) =>
this.toEntry(baseUrl, template, {
slug: row.slug,
updatedAt: row.updated_at,
}),
);
}
private async loadPublishedPortfolios(
businessId: bigint,
baseUrl: string,
template: string,
): Promise<SitemapUrlEntry[]> {
const rows = await this.prisma.portfolios.findMany({
where: {
business_id: businessId,
status: ContentStatus.published,
},
select: {
slug: true,
updated_at: true,
},
orderBy: [{ updated_at: 'desc' }, { id: 'desc' }],
});
return rows.map((row) =>
this.toEntry(baseUrl, template, {
slug: row.slug,
updatedAt: row.updated_at,
}),
);
}
private toEntry(
baseUrl: string,
template: string,
row: PublishedSlugRow,
): SitemapUrlEntry {
const path = applyPathTemplate(template, row.slug);
return {
loc: buildAbsoluteUrl(baseUrl, path),
lastmod: row.updatedAt,
changefreq: 'weekly',
priority: 0.7,
};
}
}
+30
View File
@@ -85,6 +85,36 @@ Use `slot.key` in the placeholder. Match `pageKey` to the website page. For a si
`kind` is `single` (one image) or `list` (duplicatable). For a fixed row, set `itemCount` (e.g. `2`). Leave `itemCount` null for an unbounded slider. `aspectRatio` must look like `16:9`. Do not invent CMS/upload APIs.
### Sitemap (SEO)
Meshkee generates `https://<WEBSITE_DOMAIN>/sitemap.xml` on the API from published CMS content plus a manifest your site publishes. Nginx on the storefront proxies `/sitemap.xml` and `/robots.txt` to the API — do not hardcode a static sitemap file in the repo unless you know nginx is not proxying yet.
**Publish route manifest** so the business dashboard can sync static pages and URL templates:
`GET https://<WEBSITE_DOMAIN>/meshkee/sitemap-config.json`
```json
{
"baseUrl": "https://sanihome.ir",
"staticPages": [
{ "path": "/", "changefreq": "daily", "priority": 1.0 },
{ "path": "/about", "priority": 0.6 },
{ "path": "/contact", "priority": 0.5 },
{ "path": "/blog", "priority": 0.7 }
],
"templates": {
"product": "/products/{slug}",
"blog": "/blog/{slug}",
"portfolio": "/portfolios/{slug}"
}
}
```
- `path` must start with `/`. Do not include checkout, login, or customer-dashboard URLs.
- Each `templates` value must include `{slug}` and match how this site routes detail pages.
- After deploy, the business owner syncs from **Website → Settings** in the dashboard.
- Dynamic URLs (products, blogs, portfolios) come from the CMS; only paths/templates are declared here.
If OpenAPI and this brief conflict, **OpenAPI wins**.
---
+78
View File
@@ -24,6 +24,9 @@
{
"name": "Tenant"
},
{
"name": "SEO"
},
{
"name": "Homepage"
},
@@ -186,6 +189,81 @@
}
}
},
"/tenants/{domain}/sitemap.xml": {
"get": {
"tags": [
"SEO"
],
"summary": "XML sitemap for search engines",
"description": "Published products, blogs, and portfolios for this tenant (respects enabled modules), plus static pages and URL templates from the synced website manifest (`GET https://{domain}/meshkee/sitemap-config.json`). Default paths when not synced: `/`, `/products/{slug}`, `/blog/{slug}`, `/portfolios/{slug}`. Proxied from `https://{domain}/sitemap.xml` on the storefront. Cached in Redis; refreshed after CMS writes and manifest sync.",
"parameters": [
{
"$ref": "#/components/parameters/domain"
}
],
"responses": {
"200": {
"description": "Sitemap XML (urlset)",
"content": {
"application/xml": {
"schema": {
"type": "string"
}
}
},
"headers": {
"Cache-Control": {
"schema": {
"type": "string",
"example": "public, max-age=900"
}
}
}
},
"404": {
"description": "Unknown domain"
}
}
}
},
"/tenants/{domain}/robots.txt": {
"get": {
"tags": [
"SEO"
],
"summary": "robots.txt with sitemap reference",
"description": "Plain-text robots file pointing crawlers to `https://{domain}/sitemap.xml`. Proxied from `https://{domain}/robots.txt` on the storefront.",
"parameters": [
{
"$ref": "#/components/parameters/domain"
}
],
"responses": {
"200": {
"description": "robots.txt",
"content": {
"text/plain": {
"schema": {
"type": "string",
"example": "User-agent: *\\nAllow: /\\n\\nSitemap: https://example.com/sitemap.xml\\n"
}
}
},
"headers": {
"Cache-Control": {
"schema": {
"type": "string",
"example": "public, max-age=900"
}
}
}
},
"404": {
"description": "Unknown domain"
}
}
}
},
"/tenants/{domain}/website/business-info": {
"get": {
"tags": [