{"id":30984,"date":"2026-11-17T10:23:03","date_gmt":"2026-11-17T02:23:03","guid":{"rendered":"https:\/\/www.ixpower.cn\/?p=30984"},"modified":"2026-11-17T10:23:03","modified_gmt":"2026-11-17T02:23:03","slug":"aiops-%e5%ae%9e%e6%88%98%ef%bc%9a%e5%9f%ba%e4%ba%8e-prometheus-%e5%8e%86%e5%8f%b2%e9%a2%84%e6%b5%8b%ef%bc%8c%e8%bf%90%e7%bb%b4%e7%94%a8-deepseek-%e5%8a%a8%e6%80%81%e4%bf%ae%e6%ad%a3%e5%91%8a%e8%ad%a6","status":"publish","type":"post","link":"https:\/\/www.ixpower.cn\/index.php\/2026\/11\/17\/30984\/","title":{"rendered":"AIOps \u5b9e\u6218\uff1a\u57fa\u4e8e Prometheus \u5386\u53f2\u9884\u6d4b\uff0c\u8fd0\u7ef4\u7528 DeepSeek \u52a8\u6001\u4fee\u6b63\u544a\u8b66\u89c4\u5219\uff0c\u5e26 Diff \u4e0e\u56de\u6eda\u673a\u5236"},"content":{"rendered":"<div class=\"article-desc\">CPU \u624d\u5230 20% \u5c31\u54cd\uff1f\u544a\u8b66\u9608\u503c\u4e0d\u8be5\u88ab\u56fa\u5b9a\u8bbe\u7f6e\u3002<\/div>\n<div id=\"postspictures\" class=\"article-content\">\n<div id=\"container\" class=\"container am-engine\" data-v-01a18e2f=\"\" data-element=\"root\">\n<p>\u6211\u662f\u97e9\u5148\u8d85\uff0c51CTO \u5b66\u5802 K8s\u3001Python \u6559\u5b66\u603b\u76d1\u3001AIOps \u5b9e\u6218\u8bad\u7ec3\u8425\u8bb2\u5e08\uff0c\u4e91\u8ba1\u7b97\u67b6\u6784\u5e08\uff0c\u5177\u6709 8 \u5e74\u9879\u76ee\u5b9e\u6218\u7ecf\u9a8c+5 \u5e74\u6559\u5b66\u7ecf\u9a8c\uff0c\u7ebf\u4e0a\u5b66\u5458\u5df2\u8fbe 300w+\u3002<\/p>\n<p>\u51cc\u6668\u4e24\u70b9\uff0c\u7fa4\u91cc\u53c8\u70b8\u4e86\u3002CPU \u4f7f\u7528\u7387\u624d\u4e8c\u5341\u591a\uff0c\u5185\u5b58\u4e5f\u6ca1\u6ee1\uff0c\u78c1\u76d8\u8ddf\u7740\u53eb\u3002\u6211\u70b9\u5f00\u00a0rules.yml\uff0c\u53d1\u73b0\u9608\u503c\u8fd8\u505c\u5728\u5f88\u4e45\u4ee5\u524d\u5199\u7684 20%\u3002<img decoding=\"async\" title=\"\u56fe\u7247\" src=\"https:\/\/s2.51cto.com\/oss\/202611\/17\/8476af333bc115560220479f59377303c71ff2.webp\" alt=\"\u56fe\u7247\" data-type=\"inline\" \/><\/p>\n<p>\u53cd\u8fc7\u6765\u4e5f\u5e38\u89c1\u3002\u4e1a\u52a1\u91cf\u6da8\u4e86\u4e00\u4e2a\u5b63\u5ea6\uff0c\u9608\u503c\u8fd8\u662f\u53bb\u5e74\u7684\u6570\u3002\u771f\u8981\u9876\u6ee1\u7684\u65f6\u5019\uff0c\u544a\u8b66\u6765\u5f97\u53c8\u665a\u53c8\u731b\u3002<\/p>\n<p><img decoding=\"async\" title=\"\u56fe\u7247\" src=\"https:\/\/s2.51cto.com\/oss\/202611\/17\/35ad91305a38097debd761e9b44886826d8bcd.webp\" alt=\"\u56fe\u7247\" data-type=\"inline\" \/><\/p>\n<p>\u672c\u6587\u91cc\u6d89\u53ca\u5230\u7684\u65b9\u6848\u505a\u7684\u4e8b\u5f88\u5177\u4f53\uff1a<strong>\u5411 Prometheus \u8981\u5386\u53f2\u8bb0\u5f55\uff0c\u7b97\u51fa\u672a\u6765 10 \u5206\u949f\u7684\u6c34\u4f4d\uff0c\u8ba9 DeepSeek \u6539\u51fa\u4e00\u4efd\u80fd\u6267\u884c\u7684\u89c4\u5219\u3002\u4eba\u770b\u5b8c diff\uff0c\u624d\u5141\u8bb8\u5b83\u53bb\u78b0\u670d\u52a1\u5668\u4e0a\u7684\u6587\u4ef6\u3002<\/strong><\/p>\n<p><img decoding=\"async\" title=\"\u56fe\u7247\" src=\"https:\/\/s2.51cto.com\/oss\/202611\/17\/97f283323e870a62212211b1adf4f8eb98e0d7.webp\" alt=\"\u56fe\u7247\" data-type=\"inline\" \/><\/p>\n<h2>\u5148\u628a\u6c34\u4f4d\u7b97\u51fa\u6765<\/h2>\n<p>\u5728<strong>\u6293\u53d6\u5386\u53f2\u6570\u636e\u9875\u9762<\/strong>\u4e0a\uff0c\u9009\u62e9\u300c\u8fc7\u53bb 1 \u5929\u300d\uff0c\u91c7\u6837\u6b65\u957f\u81ea\u52a8\u662f 1 \u5206\u949f\u30029 \u6761\u73b0\u6709\u544a\u8b66\u4e00\u8d77\u62c9\u53d6\uff0c\u6bcf\u6761\u5e8f\u5217\u7ed9\u51fa\u6700\u5c0f\u3001\u5e73\u5747\u3001P95\u3001\u6700\u65b0\u503c\u3001\u659c\u7387\u548c 10 \u5206\u949f\u9884\u6d4b\u503c\u3002<\/p>\n<p>\u8fd0\u7ef4\u770b\u00a0<code>node-a<\/code>\u00a0\u8fd9\u4e00\u884c\u5c31\u591f\u4e86\u3002CPU \u6700\u65b0 50.9\uff0cP95 57.0\uff0c10 \u5206\u949f\u9884\u6d4b 49.5\u3002\u89c4\u5219\u91cc\u5982\u679c\u8fd8\u5199\u7740\u00a0<code>&gt; 20<\/code>\uff0c\u8fd9\u6761\u544a\u8b66\u4f1a\u4ece\u65e9\u54cd\u5230\u665a\u3002<\/p>\n<p>\u67e5\u8be2\u4e0d\u662f\u53e6\u5199\u4e00\u5957\u6307\u6807\uff0c\u5c31\u662f\u89c4\u5219\u91cc\u539f\u6765\u7684 PromQL\u3002CPU \u548c\u5185\u5b58\u662f\u8fd9\u6837\u53d6\u7684\uff1a<\/p>\n<div>\n<div class=\"hljs-cto\">\n<div class=\"hljs-cto\"><button class=\"copy_btn disable\" data-clipboard-target=\"#code_id_0\">\u590d\u5236<\/button><\/p>\n<div class=\"code-toolbar\">\n<pre class=\"has-pre-numbering language-plain\" tabindex=\"0\"><code class=\"language-plain\">{\r\n    \"alert\": \"\u7269\u7406\u8282\u70b9cpu\u4f7f\u7528\u7387\",\r\n    \"unit\": \"%\",\r\n    \"direction\": \"higher_is_worse\",\r\n    \"query\": '(100 - avg(irate(node_cpu_seconds_total{mode=\"idle\"}[5m])) by (instance) * 100)',\r\n},\r\n{\r\n    \"alert\": \"\u7269\u7406\u8282\u70b9\u5185\u5b58\u4f7f\u7528\u7387\",\r\n    \"unit\": \"%\",\r\n    \"direction\": \"higher_is_worse\",\r\n    \"query\": (\r\n        \"(node_memory_MemTotal_bytes - (node_memory_MemFree_bytes + \"\r\n        \"node_memory_Buffers_bytes + node_memory_Cached_bytes)) \/ \"\r\n        \"node_memory_MemTotal_bytes * 100\"\r\n    ),\r\n},<\/code><\/pre>\n<ul id=\"code_id_0\" class=\"pre-numbering\">\n<li>1.<\/li>\n<li>2.<\/li>\n<li>3.<\/li>\n<li>4.<\/li>\n<li>5.<\/li>\n<li>6.<\/li>\n<li>7.<\/li>\n<li>8.<\/li>\n<li>9.<\/li>\n<li>10.<\/li>\n<li>11.<\/li>\n<li>12.<\/li>\n<li>13.<\/li>\n<li>14.<\/li>\n<li>15.<\/li>\n<li>16.<\/li>\n<\/ul>\n<div class=\"toolbar\"><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<p><strong>\u62c9\u6570\u8d70 Prometheus \u7684\u533a\u95f4\u63a5\u53e3\u3002<\/strong>1 \u5929\u7528 60 \u79d2\u4e00\u6b65\uff0c7 \u5929\u7528 5 \u5206\u949f\uff0c1 \u4e2a\u6708\u7528 30 \u5206\u949f\uff0c1 \u5e74\u7528 6 \u5c0f\u65f6\u3002\u4fee\u6539\u4e0b\u4e00\u6863\u9608\u503c\uff0c\u4f18\u5148\u770b 1 \u5929\u6216 7 \u5929\u7684\u6570\u636e\u3002<\/p>\n<div>\n<div class=\"hljs-cto\">\n<div class=\"hljs-cto\"><button class=\"copy_btn disable\" data-clipboard-target=\"#code_id_1\">\u590d\u5236<\/button><\/p>\n<div class=\"code-toolbar\">\n<pre class=\"has-pre-numbering language-plain\" tabindex=\"0\"><code class=\"language-plain\">def query_range(base, query, start, end, step):\r\n    response = requests.get(\r\n        f\"{base}\/api\/v1\/query_range\",\r\n        params={\"query\": query, \"start\": start, \"end\": end, \"step\": step},\r\n        timeout=(5, 90),\r\n    )\r\n    response.raise_for_status()\r\n    body = response.json()\r\n    if body.get(\"status\") != \"success\":\r\n        raise RuntimeError(body.get(\"error\") or \"Prometheus \u67e5\u8be2\u5931\u8d25\")\r\n    return body.get(\"data\", {}).get(\"result\") or []<\/code><\/pre>\n<ul id=\"code_id_1\" class=\"pre-numbering\">\n<li>1.<\/li>\n<li>2.<\/li>\n<li>3.<\/li>\n<li>4.<\/li>\n<li>5.<\/li>\n<li>6.<\/li>\n<li>7.<\/li>\n<li>8.<\/li>\n<li>9.<\/li>\n<li>10.<\/li>\n<li>11.<\/li>\n<\/ul>\n<div class=\"toolbar\"><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<p><strong>10 \u5206\u949f\u9884\u6d4b\u503c\u4e0d\u662f\u6a21\u578b\u5148\u731c\u7684\u3002<\/strong>\u7a0b\u5e8f\u5148\u7528\u6700\u8fd1\u5927\u7ea6 30 \u5206\u949f\u7684\u659c\u7387\u505a\u4e00\u6761\u5916\u63a8\uff0c\u8fde\u540c P95\u3001\u6700\u65b0\u503c\u4e00\u8d77\u4ea4\u7ed9\u6a21\u578b\u3002\u6a21\u578b\u8d1f\u8d23\u51b3\u5b9a\u9608\u503c\uff0c\u4e0d\u8d1f\u8d23\u91cd\u7b97\u8fd9\u5f20\u8868\u3002<\/p>\n<div>\n<div class=\"hljs-cto\">\n<div class=\"hljs-cto\"><button class=\"copy_btn disable\" data-clipboard-target=\"#code_id_2\">\u590d\u5236<\/button><\/p>\n<div class=\"code-toolbar\">\n<pre class=\"has-pre-numbering language-plain\" tabindex=\"0\"><code class=\"language-plain\">slope = recent_slope_per_min(times, values)\r\nlast = values[-1]\r\nreturn {\r\n    \"p95\": round_num(series.quantile(0.95)),\r\n    \"last\": round_num(last),\r\n    \"slope_per_min\": round_num(slope),\r\n    \"predict_10min\": round_num(last + slope * 10),\r\n}\r\n\u5207\u5230 CPU \u66f2\u7ebf\uff0c\u4e24\u53f0\u673a\u5668\u4e00\u6574\u5929\u90fd\u5728 40 \u4e0a\u4e0b\u6643\uff0c\u6ca1\u6709\u4e00\u6761\u9760\u8fd1 20\u3002<\/code><\/pre>\n<ul id=\"code_id_2\" class=\"pre-numbering\">\n<li>1.<\/li>\n<li>2.<\/li>\n<li>3.<\/li>\n<li>4.<\/li>\n<li>5.<\/li>\n<li>6.<\/li>\n<li>7.<\/li>\n<li>8.<\/li>\n<li>9.<\/li>\n<\/ul>\n<div class=\"toolbar\"><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<h2>\u6a21\u578b\u4ea4\u51fa\u7684\u662f\u89c4\u5219\uff0c\u4e0d\u662f\u4e00\u6bb5\u5efa\u8bae<\/h2>\n<p><img decoding=\"async\" title=\"\u56fe\u7247\" src=\"https:\/\/s2.51cto.com\/oss\/202611\/17\/8244927654bc45f2e818041611b7c515dcb78b.webp\" alt=\"\u56fe\u7247\" data-type=\"inline\" \/><\/p>\n<p>\u70b9\u51fb\u300c\u751f\u6210\u672a\u6765 10 \u5206\u949f\u544a\u8b66\u89c4\u5219\u300d\u4e4b\u540e\uff0c\u7ed3\u679c\u5148\u653e\u8fdb\u7f16\u8f91\u6846\uff0c\u670d\u52a1\u5668\u4e0a\u7684\u6587\u4ef6\u4e0d\u4f1a\u81ea\u52a8\u8986\u76d6\u3002<\/p>\n<p>\u8fd9\u4e00\u6b21\u5b83\u505a\u4e86\u56db\u4ef6\u4e8b\uff1a<strong>CPU \u9608\u503c\u4ece 20 \u8c03\u5230 65\uff0c<\/strong><code><strong>for<\/strong><\/code><strong>\u00a0\u4ece 2 \u79d2\u6539\u6210 1 \u5206\u949f\uff1b\u5185\u5b58\u4ece 20 \u8c03\u5230 75\uff1b\u78c1\u76d8\u5bb9\u91cf\u4ece 80 \u8c03\u5230 88\u3002<\/strong><\/p>\n<p>\u8fd9\u91cc Nginx\u3001\u6d41\u91cf\u3001TCP \u6ca1\u52a8\uff0c<code>InstanceDown<\/code>\u00a0\u4ecd\u662f\u00a0<code>up == 0<\/code>\u3002<\/p>\n<p>\u6a21\u578b\u88ab\u8981\u6c42\u53ea\u8fd4\u56de\u4e00\u4e2a JSON\uff0c\u91cc\u9762\u662f\u603b\u7ed3\u3001\u9884\u6d4b\u8bf4\u660e\u3001\u6539\u52a8\u5217\u8868\u548c\u5b8c\u6574\u7684\u00a0<code>rules.yml<\/code>\u3002\u7ea6\u675f\u5199\u5728\u7cfb\u7edf\u63d0\u793a\u91cc\uff0c\u5173\u952e\u5c31\u8fd9\u51e0\u6761\uff1a<\/p>\n<ol data-id=\"o9d5df12-WXYX9kTP\">\n<li data-id=\"ld70c578-nNqGrD8e\">rules_yml \u5fc5\u987b\u662f\u5b8c\u6574 YAML\uff0c\u4fdd\u7559\u539f\u6709 groups\u3001\u544a\u8b66\u540d\u79f0\u548c\u6807\u7b7e\u3002<\/li>\n<li data-id=\"ld70c578-kP3WPN8T\">\u53ea\u8c03\u6574\u6709\u5386\u53f2\u6570\u636e\u4e14 adjustable \u4e3a true \u7684\u9608\u503c\u3002<\/li>\n<li data-id=\"ld70c578-HTtuKRHL\">InstanceDown \u7684 expr \u5fc5\u987b\u4fdd\u6301 up == 0\u3002<\/li>\n<li data-id=\"ld70c578-s7zvnU74\"><strong>\u8d8a\u9ad8\u8d8a\u7cdf<\/strong>\uff1a\u9608\u503c\u8981\u9ad8\u4e8e\u672a\u6765 10 \u5206\u949f\u9884\u6d4b\u503c\u548c\u8fd1\u671f P95\uff0c\u7559\u51fa\u4f59\u91cf\u3002\u767e\u5206\u6bd4\u5fc5\u987b\u5728 0 \u5230 100 \u4e4b\u95f4\u3002<\/li>\n<li data-id=\"ld70c578-CYMjVwAs\"><strong>\u8d8a\u4f4e\u8d8a\u7cdf<\/strong>\uff1a\u9608\u503c\u8981\u4f4e\u4e8e\u672a\u6765 10 \u5206\u949f\u9884\u6d4b\u503c\uff0c\u540c\u6837\u7559\u51fa\u4f59\u91cf\u3002<\/li>\n<li data-id=\"ld70c578-AutsFVX5\">\u9884\u6d4b\u7a97\u53e3\u662f 10 \u5206\u949f\uff0c\u4e0d\u4ee3\u8868 for \u8981\u6539\u6210 10m\u3002\u566a\u58f0\u5927\u65f6\uff0c\u53ef\u4ee5\u628a for \u8c03\u5230 30s \u81f3 2m\u3002<\/li>\n<li data-id=\"ld70c578-BzddM0yF\">\u4e0d\u8981\u65b0\u589e\u544a\u8b66\uff0c\u4e0d\u8981\u5220\u9664\u544a\u8b66\uff0c\u4e0d\u8981\u4fee\u6539\u6307\u6807\u540d\u79f0\u3002<\/li>\n<\/ol>\n<ul data-id=\"ua73dd4b-Uc0mjMPZ\">\n<li data-id=\"ld70c578-G96hBWXZ\">CPU \u5c5e\u4e8e\u8d8a\u9ad8\u8d8a\u7cdf\u3002P95 \u5df2\u7ecf\u5230 57\uff0c\u9884\u6d4b\u5728 50 \u9644\u8fd1\uff0c20 \u4e00\u5b9a\u8bef\u62a5\uff0c65 \u8fd8\u7559\u7740\u5c16\u5cf0\u80fd\u89e6\u53d1\u7684\u4f4d\u7f6e\u3002<code>for: 2s<\/code>\u00a0\u4f1a\u88ab\u6bdb\u523a\u6253\u54cd\uff0c\u6240\u4ee5\u4e00\u8d77\u6539\u6210\u00a0<code>1m<\/code>\u3002<\/li>\n<li data-id=\"ld70c578-HK7Ngj64\">\u78c1\u76d8 IO \u90a3\u6761\u76f8\u53cd\uff0c\u8868\u8fbe\u5f0f\u662f\u300c100 \u51cf IO \u5360\u7528\u300d\uff0c\u6570\u503c\u8d8a\u4f4e\u8d8a\u5dee\uff0c\u9608\u503c\u8981\u5f80\u9884\u6d4b\u503c\u4e0b\u9762\u653e\uff0c\u4e0d\u80fd\u7167\u7740 CPU \u7684\u65b9\u5411\u62ac\u3002<\/li>\n<\/ul>\n<p><img data-dominant-color=\"cbcdcc\" data-has-transparency=\"false\" style=\"--dominant-color: #cbcdcc;\" loading=\"lazy\" decoding=\"async\" class=\"not-transparent alignnone size-full wp-image-30985\" src=\"https:\/\/aiforumimage.oss-cn-shanghai.aliyuncs.com\/wp-content\/uploads\/2026\/11\/20261117022306727.png\" width=\"513\" height=\"849\" srcset=\"https:\/\/aiforumimage.oss-cn-shanghai.aliyuncs.com\/wp-content\/uploads\/2026\/11\/20261117022306727.png 513w, https:\/\/aiforumimage.oss-cn-shanghai.aliyuncs.com\/wp-content\/uploads\/2026\/11\/20261117022306727-181x300.png 181w, https:\/\/aiforumimage.oss-cn-shanghai.aliyuncs.com\/wp-content\/uploads\/2026\/11\/20261117022306727-91x150.png 91w\" sizes=\"auto, (max-width: 513px) 100vw, 513px\" \/><\/p>\n<h2>\u4eba\u770b\u5b8c diff\uff0c\u624d\u5141\u8bb8\u5199\u8fdb\u53bb<\/h2>\n<p>\u89c4\u5219\u4e0b\u53d1\u9875\u5de6\u8fb9\u662f\u5f85\u4e0b\u53d1\u7684\u5168\u6587\uff0c\u4e0b\u9762\u662f\u548c\u8f7d\u5165\u65f6\u5185\u5bb9\u7684\u5bf9\u6bd4\u3002YAML \u4e0d\u5408\u6cd5\uff0c\u5199\u5165\u6309\u94ae\u662f\u7070\u7684\uff0c\u8fd8\u8981\u52fe\u4e0a\u300c\u6211\u5df2\u6838\u5bf9\u5dee\u5f02\u300d\u3002<\/p>\n<p>\u8fd9\u6b21 diff \u91cc\u7ea2\u7684\u662f\u65e7\u9608\u503c\uff0c\u7eff\u7684\u662f\u65b0\u9608\u503c\u3002\u544a\u8b66\u540d\u3001\u6807\u7b7e\u3001PromQL \u7684\u6307\u6807\u90fd\u6ca1\u88ab\u6362\u6389\u3002<\/p>\n<p>\u5bf9\u5e94\u5230\u6587\u4ef6\u91cc\uff0cCPU \u8fd9\u4e00\u6bb5\u5c31\u662f\uff1a<\/p>\n<div>\n<div class=\"hljs-cto\">\n<div class=\"hljs-cto\"><button class=\"copy_btn disable\" data-clipboard-target=\"#code_id_3\">\u590d\u5236<\/button><\/p>\n<div class=\"code-toolbar\">\n<pre class=\"has-pre-numbering language-plain\" tabindex=\"0\"><code class=\"language-plain\">- alert: \u7269\u7406\u8282\u70b9cpu\u4f7f\u7528\u7387\r\n  expr: (100 - avg(irate(node_cpu_seconds_total{mode='idle'}[5m])) by(instance)*100)\r\n    &gt; 65\r\n  for: 1m<\/code><\/pre>\n<ul id=\"code_id_3\" class=\"pre-numbering\">\n<li>1.<\/li>\n<li>2.<\/li>\n<li>3.<\/li>\n<li>4.<\/li>\n<\/ul>\n<div class=\"toolbar\"><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<p>\u539f\u6765\u662f\u00a0<code>&gt; 20<\/code>\u00a0\u548c\u00a0<code>for: 2s\uff0c\u8fd9\u91cc\u5df2\u505a\u4fee\u6539<\/code>\u3002<\/p>\n<p><strong>\u5199\u5165\u8fdc\u7a0b\u6587\u4ef6\u65f6\uff0c\u7a0b\u5e8f\u4e0d\u76f4\u63a5\u8986\u76d6\u3002<\/strong>\u5148\u628a\u5019\u9009\u5185\u5bb9\u653e\u5230\u00a0<code>rules.yml.candidate<\/code>\uff0c\u540c\u76ee\u5f55\u6709\u00a0<code>promtool<\/code>\u00a0\u5c31\u5148\u68c0\u67e5\uff1b\u68c0\u67e5\u5931\u8d25\u5c31\u5220\u6389\u5019\u9009\uff0c\u4e0d\u81ea\u52a8\u8986\u76d6\u6b63\u5f0f\u6587\u4ef6\u3002<\/p>\n<p>\u901a\u8fc7\u4e4b\u540e\uff0c\u539f\u6587\u4ef6\u590d\u5236\u6210\u00a0<code>rules.yml.bak. \u65f6\u95f4\u6233<\/code>\uff0c\u518d\u66ff\u6362\u3002<\/p>\n<div>\n<div class=\"hljs-cto\">\n<div class=\"hljs-cto\"><button class=\"copy_btn disable\" data-clipboard-target=\"#code_id_4\">\u590d\u5236<\/button><\/p>\n<div class=\"code-toolbar\">\n<pre class=\"has-pre-numbering language-plain\" tabindex=\"0\"><code class=\"language-plain\">promtool = remote_parent(path).rstrip(\"\/\") + \"\/promtool\"\r\ncheck_cmd = (\r\n    f\"if [ -x {shlex.quote(promtool)} ]; then \"\r\n    f\"{shlex.quote(promtool)} check rules {shlex.quote(staged)}; \"\r\n    \"else echo PROMTOOL_MISSING; fi\"\r\n)\r\n# \u68c0\u67e5\u5931\u8d25\uff1a\u5220\u9664\u5019\u9009\u6587\u4ef6\uff0c\u4e0d\u8986\u76d6 rules.yml\r\nstamp = datetime.now(CST).strftime(\"%Y%m%d%H%M%S\")\r\nbackup = f\"{path}.bak.{stamp}\"\r\nreplace_cmd = (\r\n    f\"if [ -f {shlex.quote(path)} ]; then cp -a {shlex.quote(path)} {shlex.quote(backup)}; \"\r\n    f\"else echo NO_ORIGINAL; fi &amp;&amp; mv -f {shlex.quote(staged)} {shlex.quote(path)}\"\r\n)<\/code><\/pre>\n<ul id=\"code_id_4\" class=\"pre-numbering\">\n<li>1.<\/li>\n<li>2.<\/li>\n<li>3.<\/li>\n<li>4.<\/li>\n<li>5.<\/li>\n<li>6.<\/li>\n<li>7.<\/li>\n<li>8.<\/li>\n<li>9.<\/li>\n<li>10.<\/li>\n<li>11.<\/li>\n<li>12.<\/li>\n<li>13.<\/li>\n<\/ul>\n<div class=\"toolbar\"><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<p>\u5199\u5b8c\u53ef\u4ee5\u8c03\u7528 Prometheus \u7684 \/-\/reload\u3002\u5982\u679c\u9009\u9519\u4e86\uff0c\u4ece rules.yml.bak.* \u91cc\u6311\u4e00\u4efd\u8986\u76d6\u56de\u53bb\uff0c\u518d\u52a0\u8f7d\u4e00\u6b21\u3002<\/p>\n<h2>\u8fd9\u9875\u8fd8\u76ef\u7740\u54ea\u4e9b\u544a\u8b66<\/h2>\n<p>\u9664\u4e86 CPU\u3001\u5185\u5b58\u548c\u78c1\u76d8\u5bb9\u91cf\uff0c\u540c\u4e00\u4efd\u89c4\u5219\u91cc\u8fd8\u6709\uff1a<\/p>\n<ul data-id=\"ua73dd4b-VkEh84mi\">\n<li data-id=\"ld70c578-EH0URDVO\">Nginx 2xx \u8bf7\u6c42\u901f\u7387<\/li>\n<li data-id=\"ld70c578-ahL0pXOM\">\u5b9e\u4f8b\u662f\u5426\u5b58\u6d3b\uff0c\u53ea\u89c2\u5bdf\uff0c\u4e0d\u6539\u00a0<code>up == 0<\/code><\/li>\n<li data-id=\"ld70c578-h5p3ggRU\">\u78c1\u76d8 IO \u6027\u80fd\uff0c\u8d8a\u4f4e\u8d8a\u7cdf<\/li>\n<li data-id=\"ld70c578-PMekNYIa\">\u5165\u7f51\u3001\u51fa\u7f51\u6d41\u91cf<\/li>\n<li data-id=\"ld70c578-cYEARCIF\">TCP \u5f53\u524d\u8fde\u63a5\u6570<\/li>\n<\/ul>\n<p>\u81ea\u5b9a\u4e49 PromQL \u53ef\u4ee5\u52a0\u8fdb\u6765\u770b\u66f2\u7ebf\uff0c\u4f46\u4e0d\u4f1a\u56e0\u6b64\u591a\u51fa\u4e00\u6761\u65b0\u544a\u8b66\u3002\u53ef\u9009\u4e00\u6b65\uff1a<\/p>\n<p><strong>\u628a\u6700\u65b0\u503c\u548c 10 \u5206\u949f\u9884\u6d4b\u63a8\u5230 Pushgateway\uff0cjob \u540d\u662f\u00a0<\/strong><code><strong>deepseek_prediction<\/strong><\/code><strong>\uff0c\u6307\u6807\u662f\u00a0<\/strong><code><strong>deepseek_metric_last<\/strong><\/code><strong>\u00a0\u548c\u00a0<\/strong><code><strong>deepseek_metric_predict_10min<\/strong><\/code><strong>\u3002<\/strong><\/p>\n<p>Grafana \u4e0a\u5c31\u80fd\u770b\u5230\u6a21\u578b\u5f53\u65f6\u62ff\u6765\u505a\u5224\u65ad\u7684\u6c34\u4f4d\u3002<\/p>\n<p><strong>\u51fa\u73b0\u544a\u8b66\uff0c\u8fd0\u7ef4\u7684\u6392\u969c\u987a\u5e8f\u5c31\u4e09\u6b65\uff1a\u5386\u53f2\u6570\u636e\uff0c\u667a\u80fd\u5206\u6790\uff0c\u89c4\u5219\u4e0b\u53d1\u3002<\/strong>\u5efa\u8bae\u5927\u5bb6\u7528\u6d4f\u89c8\u5668\u8bbf\u95ee\uff0c\u6309\u540c\u76ee\u5f55\u7684\u90e8\u7f72\u624b\u518c\u6253\u6210\u955c\u50cf\u5373\u53ef\u3002<\/p>\n<p><strong>\u6ce8\u610f\uff1a\u9608\u503c\u4f1a\u8fc7\u671f\u3002<\/strong>\u8fc7\u671f\u4e4b\u524d\uff0c\u5148\u5bf9\u6bd4\u8fc7\u53bb\u4e00\u5929\u7684 P95 \u548c\u672a\u6765 10 \u5206\u949f\u7684\u9884\u6d4b\uff0c\u518d\u51b3\u5b9a 20 \u8fd8\u8be5\u4e0d\u8be5\u7559\u5728\u89c4\u5219\u91cc\u3002<\/p>\n<\/div>\n<\/div>\n<p>&nbsp;<\/p>\n<p>\u6587\u7ae0\u6765\u81ea\uff1a51CTO<\/p>\n<div class=\"pvc_clear\"><\/div>\n<p id=\"pvc_stats_30984\" class=\"pvc_stats total_only  \" data-element-id=\"30984\" style=\"\"><i class=\"pvc-stats-icon medium\" aria-hidden=\"true\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" version=\"1.0\" viewBox=\"0 0 502 315\" preserveAspectRatio=\"xMidYMid meet\"><g transform=\"translate(0,332) scale(0.1,-0.1)\" fill=\"\" stroke=\"none\"><path d=\"M2394 3279 l-29 -30 -3 -207 c-2 -182 0 -211 15 -242 39 -76 157 -76 196 0 15 31 17 60 15 243 l-3 209 -33 29 c-26 23 -41 29 -80 29 -41 0 -53 -5 -78 -31z\"\/><path d=\"M3085 3251 c-45 -19 -58 -50 -96 -229 -47 -217 -49 -260 -13 -295 52 -53 146 -42 177 20 16 31 87 366 87 410 0 70 -86 122 -155 94z\"\/><path d=\"M1751 3234 c-13 -9 -29 -31 -37 -50 -12 -29 -10 -49 21 -204 19 -94 39 -189 45 -210 14 -50 54 -80 110 -80 34 0 48 6 76 34 21 21 34 44 34 59 0 14 -18 113 -40 219 -37 178 -43 195 -70 221 -36 32 -101 37 -139 11z\"\/><path d=\"M1163 3073 c-36 -7 -73 -59 -73 -102 0 -56 133 -378 171 -413 34 -32 83 -37 129 -13 70 36 67 87 -16 290 -86 209 -89 214 -129 231 -35 14 -42 15 -82 7z\"\/><path d=\"M3689 3066 c-15 -9 -33 -30 -42 -48 -48 -103 -147 -355 -147 -375 0 -98 131 -148 192 -74 13 15 57 108 97 206 80 196 84 226 37 273 -30 30 -99 39 -137 18z\"\/><path d=\"M583 2784 c-38 -19 -67 -74 -58 -113 9 -42 211 -354 242 -373 16 -10 45 -18 66 -18 51 0 107 52 107 100 0 39 -1 41 -124 234 -80 126 -108 162 -133 173 -41 17 -61 16 -100 -3z\"\/><path d=\"M4250 2784 c-14 -9 -74 -91 -133 -183 -95 -150 -107 -173 -107 -213 0 -55 33 -94 87 -104 67 -13 90 8 211 198 130 202 137 225 78 284 -27 27 -42 34 -72 34 -22 0 -50 -8 -64 -16z\"\/><path d=\"M2275 2693 c-553 -48 -1095 -270 -1585 -649 -135 -104 -459 -423 -483 -476 -23 -49 -22 -139 2 -186 73 -142 361 -457 571 -626 285 -228 642 -407 990 -497 242 -63 336 -73 660 -74 310 0 370 5 595 52 535 111 1045 392 1455 803 122 121 250 273 275 326 19 41 19 137 0 174 -41 79 -309 363 -465 492 -447 370 -946 591 -1479 653 -113 14 -422 18 -536 8z m395 -428 c171 -34 330 -124 456 -258 112 -119 167 -219 211 -378 27 -96 24 -300 -5 -401 -72 -255 -236 -447 -474 -557 -132 -62 -201 -76 -368 -76 -167 0 -236 14 -368 76 -213 98 -373 271 -451 485 -162 444 86 934 547 1084 153 49 292 57 452 25z m909 -232 c222 -123 408 -262 593 -441 76 -74 138 -139 138 -144 0 -16 -233 -242 -330 -319 -155 -123 -309 -223 -461 -299 l-81 -41 32 46 c18 26 49 83 70 128 143 306 141 649 -6 957 -25 52 -61 116 -79 142 l-34 47 45 -20 c26 -10 76 -36 113 -56z m-2057 25 c-40 -58 -105 -190 -130 -263 -110 -324 -59 -707 132 -981 25 -35 42 -64 37 -64 -19 0 -241 119 -326 174 -188 122 -406 314 -532 468 l-58 71 108 103 c185 178 428 349 672 473 66 33 121 60 123 61 2 0 -10 -19 -26 -42z\"\/><path d=\"M2375 1950 c-198 -44 -350 -190 -395 -379 -18 -76 -8 -221 19 -290 114 -284 457 -406 731 -260 98 52 188 154 231 260 27 69 37 214 19 290 -38 163 -166 304 -326 360 -67 23 -215 33 -279 19z\"\/><\/g><\/svg><\/i> <img loading=\"lazy\" decoding=\"async\" width=\"16\" height=\"16\" alt=\"Loading\" src=\"https:\/\/www.ixpower.cn\/wp-content\/plugins\/page-views-count\/ajax-loader-2x.gif\" border=0 \/><\/p>\n<div class=\"pvc_clear\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>CPU \u624d\u5230 20% \u5c31\u54cd\uff1f\u544a\u8b66\u9608\u503c\u4e0d\u8be5\u88ab\u56fa\u5b9a\u8bbe\u7f6e\u3002 \u6211\u662f\u97e9\u5148\u8d85\uff0c51CTO \u5b66\u5802 K8s\u3001Python \u6559\u5b66 [&hellip;]<\/p>\n<div class=\"pvc_clear\"><\/div>\n<p id=\"pvc_stats_30984\" class=\"pvc_stats total_only  \" data-element-id=\"30984\" style=\"\"><i class=\"pvc-stats-icon medium\" aria-hidden=\"true\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" version=\"1.0\" viewBox=\"0 0 502 315\" preserveAspectRatio=\"xMidYMid meet\"><g transform=\"translate(0,332) scale(0.1,-0.1)\" fill=\"\" stroke=\"none\"><path d=\"M2394 3279 l-29 -30 -3 -207 c-2 -182 0 -211 15 -242 39 -76 157 -76 196 0 15 31 17 60 15 243 l-3 209 -33 29 c-26 23 -41 29 -80 29 -41 0 -53 -5 -78 -31z\"\/><path d=\"M3085 3251 c-45 -19 -58 -50 -96 -229 -47 -217 -49 -260 -13 -295 52 -53 146 -42 177 20 16 31 87 366 87 410 0 70 -86 122 -155 94z\"\/><path d=\"M1751 3234 c-13 -9 -29 -31 -37 -50 -12 -29 -10 -49 21 -204 19 -94 39 -189 45 -210 14 -50 54 -80 110 -80 34 0 48 6 76 34 21 21 34 44 34 59 0 14 -18 113 -40 219 -37 178 -43 195 -70 221 -36 32 -101 37 -139 11z\"\/><path d=\"M1163 3073 c-36 -7 -73 -59 -73 -102 0 -56 133 -378 171 -413 34 -32 83 -37 129 -13 70 36 67 87 -16 290 -86 209 -89 214 -129 231 -35 14 -42 15 -82 7z\"\/><path d=\"M3689 3066 c-15 -9 -33 -30 -42 -48 -48 -103 -147 -355 -147 -375 0 -98 131 -148 192 -74 13 15 57 108 97 206 80 196 84 226 37 273 -30 30 -99 39 -137 18z\"\/><path d=\"M583 2784 c-38 -19 -67 -74 -58 -113 9 -42 211 -354 242 -373 16 -10 45 -18 66 -18 51 0 107 52 107 100 0 39 -1 41 -124 234 -80 126 -108 162 -133 173 -41 17 -61 16 -100 -3z\"\/><path d=\"M4250 2784 c-14 -9 -74 -91 -133 -183 -95 -150 -107 -173 -107 -213 0 -55 33 -94 87 -104 67 -13 90 8 211 198 130 202 137 225 78 284 -27 27 -42 34 -72 34 -22 0 -50 -8 -64 -16z\"\/><path d=\"M2275 2693 c-553 -48 -1095 -270 -1585 -649 -135 -104 -459 -423 -483 -476 -23 -49 -22 -139 2 -186 73 -142 361 -457 571 -626 285 -228 642 -407 990 -497 242 -63 336 -73 660 -74 310 0 370 5 595 52 535 111 1045 392 1455 803 122 121 250 273 275 326 19 41 19 137 0 174 -41 79 -309 363 -465 492 -447 370 -946 591 -1479 653 -113 14 -422 18 -536 8z m395 -428 c171 -34 330 -124 456 -258 112 -119 167 -219 211 -378 27 -96 24 -300 -5 -401 -72 -255 -236 -447 -474 -557 -132 -62 -201 -76 -368 -76 -167 0 -236 14 -368 76 -213 98 -373 271 -451 485 -162 444 86 934 547 1084 153 49 292 57 452 25z m909 -232 c222 -123 408 -262 593 -441 76 -74 138 -139 138 -144 0 -16 -233 -242 -330 -319 -155 -123 -309 -223 -461 -299 l-81 -41 32 46 c18 26 49 83 70 128 143 306 141 649 -6 957 -25 52 -61 116 -79 142 l-34 47 45 -20 c26 -10 76 -36 113 -56z m-2057 25 c-40 -58 -105 -190 -130 -263 -110 -324 -59 -707 132 -981 25 -35 42 -64 37 -64 -19 0 -241 119 -326 174 -188 122 -406 314 -532 468 l-58 71 108 103 c185 178 428 349 672 473 66 33 121 60 123 61 2 0 -10 -19 -26 -42z\"\/><path d=\"M2375 1950 c-198 -44 -350 -190 -395 -379 -18 -76 -8 -221 19 -290 114 -284 457 -406 731 -260 98 52 188 154 231 260 27 69 37 214 19 290 -38 163 -166 304 -326 360 -67 23 -215 33 -279 19z\"\/><\/g><\/svg><\/i> <img loading=\"lazy\" decoding=\"async\" width=\"16\" height=\"16\" alt=\"Loading\" src=\"https:\/\/www.ixpower.cn\/wp-content\/plugins\/page-views-count\/ajax-loader-2x.gif\" border=0 \/><\/p>\n<div class=\"pvc_clear\"><\/div>\n","protected":false},"author":56,"featured_media":30985,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"publish_to_discourse":"0","publish_post_category":"5","wpdc_auto_publish_overridden":"","wpdc_topic_tags":"","wpdc_pin_topic":"","wpdc_pin_until":"","discourse_post_id":"","discourse_permalink":"","wpdc_publishing_response":"","wpdc_publishing_error":"","footnotes":""},"categories":[3,23,2327,80],"tags":[2361],"class_list":["post-30984","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-23","category-80","tag-aiops--prometheus--deepseek-"],"_links":{"self":[{"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/posts\/30984","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/users\/56"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/comments?post=30984"}],"version-history":[{"count":1,"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/posts\/30984\/revisions"}],"predecessor-version":[{"id":30986,"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/posts\/30984\/revisions\/30986"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/media\/30985"}],"wp:attachment":[{"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/media?parent=30984"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/categories?post=30984"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ixpower.cn\/index.php\/wp-json\/wp\/v2\/tags?post=30984"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}