|
@@ -1,36 +1,43 @@
|
|
|
-lq@lq:~/Fine-tuning$ cp backend/scripts/test_ms_api.py backend/data/ && sudo docker exec -it finetune-backend python3 /root/Fine-tuning/backend/data/test_ms_api.py tany0699/carBrands50
|
|
|
|
|
-数据集: tany0699/carBrands50
|
|
|
|
|
-
|
|
|
|
|
-=== 用 MsDataset.load() 下载 ===
|
|
|
|
|
-2026-05-28 07:26:26,648 - modelscope - INFO - No subset_name specified, defaulting to the default
|
|
|
|
|
-2026-05-28 07:26:27,880 - modelscope - WARNING - Reusing dataset dataset_builder (/root/.cache/modelscope/hub/datasets/tany0699/carBrands50/master/data_files)
|
|
|
|
|
-2026-05-28 07:26:27,880 - modelscope - INFO - Generating dataset dataset_builder (/root/.cache/modelscope/hub/datasets/tany0699/carBrands50/master/data_files)
|
|
|
|
|
-2026-05-28 07:26:27,880 - modelscope - INFO - Reusing cached meta-data file: /root/.cache/modelscope/hub/datasets/tany0699/carBrands50/master/data_files/a6ade4dacefa0beffa564bf1f50f7ffd
|
|
|
|
|
-split='train' 成功, 共 4397 条
|
|
|
|
|
-
|
|
|
|
|
-=== 处理前 5 条数据(复制图片) ===
|
|
|
|
|
-Record 0: image:FILE -> copied Bentley_025.jpg (6811 bytes) as 000000.jpg
|
|
|
|
|
-Record 1: image:FILE -> copied Bentley_004.jpg (12710 bytes) as 000001.jpg
|
|
|
|
|
-Record 2: image:FILE -> copied Bentley_038.jpg (6993 bytes) as 000002.jpg
|
|
|
|
|
-Record 3: image:FILE -> copied Bentley_011.jpg (8629 bytes) as 000003.jpg
|
|
|
|
|
-Record 4: image:FILE -> copied Bentley_006.jpg (7090 bytes) as 000004.jpg
|
|
|
|
|
-
|
|
|
|
|
-=== 写入 JSONL ===
|
|
|
|
|
-写入 5 条记录到 /tmp/ms_test_download/data.jsonl
|
|
|
|
|
-
|
|
|
|
|
-=== JSONL 内容 ===
|
|
|
|
|
-0: {"image:FILE": "images/000000.jpg", "category": 4}
|
|
|
|
|
-1: {"image:FILE": "images/000001.jpg", "category": 4}
|
|
|
|
|
-2: {"image:FILE": "images/000002.jpg", "category": 4}
|
|
|
|
|
-3: {"image:FILE": "images/000003.jpg", "category": 4}
|
|
|
|
|
-4: {"image:FILE": "images/000004.jpg", "category": 4}
|
|
|
|
|
-
|
|
|
|
|
-=== images 目录 ===
|
|
|
|
|
- 000000.jpg (6811 bytes)
|
|
|
|
|
- 000001.jpg (12710 bytes)
|
|
|
|
|
- 000002.jpg (6993 bytes)
|
|
|
|
|
- 000003.jpg (8629 bytes)
|
|
|
|
|
- 000004.jpg (7090 bytes)
|
|
|
|
|
-
|
|
|
|
|
-=== 测试完成! ===
|
|
|
|
|
-数据集目录: /tmp/ms_test_download
|
|
|
|
|
|
|
+INFO: Started server process [1]
|
|
|
|
|
+INFO: Waiting for application startup.
|
|
|
|
|
+2026-05-28 07:52:40 | INFO | peft-platform | JobQueue started with 2 workers
|
|
|
|
|
+INFO: Application startup complete.
|
|
|
|
|
+INFO: Uvicorn running on http://0.0.0.0:8010 (Press CTRL+C to quit)
|
|
|
|
|
+INFO: 172.20.0.4:35472 - "GET /api/v1/models/ HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 172.20.0.4:35474 - "GET /api/v1/datasets/ HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 172.20.0.4:35500 - "GET /api/v1/training/jobs HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 172.20.0.4:35510 - "GET /api/v1/models/ HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 172.20.0.4:35514 - "GET /api/v1/datasets/ HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 127.0.0.1:46858 - "GET /health HTTP/1.1" 200 OK
|
|
|
|
|
+2026-05-28 07:52:46 | INFO | peft-platform | Deleted dataset directory: /root/Fine-tuning/backend/data/processed/ms_tany0699_carBrands50
|
|
|
|
|
+2026-05-28 07:52:46 | INFO | peft-platform | Deleted dataset: tany0699/carBrands50
|
|
|
|
|
+INFO: 172.20.0.4:58764 - "DELETE /api/v1/datasets/98dd637f-879b-4fc4-b7ea-f64238110c25 HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 172.20.0.4:58766 - "GET /api/v1/datasets/ HTTP/1.0" 200 OK
|
|
|
|
|
+2026-05-28 07:52:51 | INFO | peft-platform | Dataset download task started: tany0699/carBrands50 (task_id=035d9cad-3c35-467b-9409-01de28520fb3)
|
|
|
|
|
+INFO: 172.20.0.4:58774 - "POST /api/v1/datasets/download HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 172.20.0.4:57858 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 172.20.0.4:57866 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+2026-05-28 07:53:00,756 - modelscope - INFO - No subset_name specified, defaulting to the default
|
|
|
|
|
+INFO: 172.20.0.4:57876 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+2026-05-28 07:53:02,365 - modelscope - WARNING - Reusing dataset dataset_builder (/root/.cache/modelscope/hub/datasets/tany0699/carBrands50/master/data_files)
|
|
|
|
|
+2026-05-28 07:53:02,365 - modelscope - INFO - Generating dataset dataset_builder (/root/.cache/modelscope/hub/datasets/tany0699/carBrands50/master/data_files)
|
|
|
|
|
+2026-05-28 07:53:02,366 - modelscope - INFO - Loading meta-data file ...
|
|
|
|
|
+4398it [00:00, 40120.82it/s]
|
|
|
|
|
+INFO: 172.20.0.4:38498 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+9% INFO: 172.20.0.4:38514 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+40% INFO: 172.20.0.4:38520 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+61% INFO: 172.20.0.4:38522 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+73% INFO: 127.0.0.1:43340 - "GET /health HTTP/1.1" 200 OK
|
|
|
|
|
+84% INFO: 172.20.0.4:47586 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+100% INFO: 172.20.0.4:47596 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 172.20.0.4:47600 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+2026-05-28 07:53:22 | INFO | peft-platform | MsDataset.load() loaded split 'train': 4397 records
|
|
|
|
|
+2026-05-28 07:53:22 | WARNING | peft-platform | MsDataset.load() failed for tany0699/carBrands50: name 'os' is not defined, falling back to CLI
|
|
|
|
|
+2026-05-28 07:53:22 | INFO | peft-platform | Fallback CLI: modelscope download --dataset tany0699/carBrands50 --local_dir /root/Fine-tuning/backend/data/processed/ms_tany0699_carBrands50
|
|
|
|
|
+INFO: 172.20.0.4:43150 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+2026-05-28 07:53:27 | INFO | peft-platform | CLI downloaded 9 files to /root/Fine-tuning/backend/data/processed/ms_tany0699_carBrands50
|
|
|
|
|
+2026-05-28 07:53:27 | INFO | peft-platform | Selected data file: /root/Fine-tuning/backend/data/processed/ms_tany0699_carBrands50/train.csv (size=140505)
|
|
|
|
|
+2026-05-28 07:53:27 | INFO | peft-platform | Dataset downloaded: tany0699/carBrands50 (4397 records)
|
|
|
|
|
+INFO: 172.20.0.4:43156 - "GET /api/v1/datasets/download/035d9cad-3c35-467b-9409-01de28520fb3 HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 172.20.0.4:43160 - "GET /api/v1/datasets/ HTTP/1.0" 200 OK
|
|
|
|
|
+INFO: 172.20.0.4:43172 - "GET /api/v1/datasets/4eb51319-31aa-4f8c-a770-0b94294d0b5a/preview?rows=10 HTTP/1.0" 200 OK
|