档案短期保存系统建设：从零搭建可落地的技术方案

一、核心架构设计

短期档案保存系统指保存期限在1-10年的电子档案管理系统，核心需求是低成本、易检索、可审计。

1.1 技术选型方案

采用分层架构：应用层（Docker容器化部署）、服务层（Spring Boot微服务）、存储层（对象存储+关系数据库）。

具体组件清单：

• 文件存储：MinIO（开源对象存储，兼容S3协议）
• 元数据存储：MySQL 8.0（社区版）
• 检索服务：Elasticsearch 7.17
• 应用框架：Spring Boot 2.7 + MyBatis Plus
• 部署环境：Docker 20.10 + Docker Compose

1.2 目录结构规划

档案存储按“年度/部门/档案类型”三级目录组织，示例：

2024/
├── HR/
│   ├── 劳动合同/
│   ├── 入职材料/
│   └── 考核记录/
├── FINANCE/
│   ├── 报销凭证/
│   └── 付款记录/
└── TECH/
├── 设计文档/
└── 测试报告/

二、环境部署与配置

2.1 基础环境安装

在CentOS 7.9服务器执行以下命令：

 安装Docker
curl -fsSL https://get.docker.com | bash -s docker
systemctl start docker
systemctl enable docker
安装Docker Compose
curl -L "https://github.com/docker/compose/releases/download/v2.20.3/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

2.2 存储服务部署

创建docker-compose.yml文件：

version: '3.8'
services:
minio:
image: minio/minio:latest
command: server /data --console-address ":9001"
ports:
- "9000:9000"
- "9001:9001"
environment:
MINIO_ROOT_USER: admin
MINIO_ROOT_PASSWORD: YourStrongPassword123!
volumes:
- ./minio_data:/data
restart: unless-stopped
mysql:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: RootPass123!
MYSQL_DATABASE: archive_db
ports:
- "3306:3306"
volumes:
- ./mysql_data:/var/lib/mysql
- ./init.sql:/docker-entrypoint-initdb.d/init.sql
restart: unless-stopped
elasticsearch:
image: elasticsearch:7.17.16
environment:
- discovery.type=single-node
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ports:
- "9200:9200"
volumes:
- ./es_data:/usr/share/elasticsearch/data
restart: unless-stopped

创建数据库初始化文件init.sql：

CREATE TABLE archive_metadata (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
file_name VARCHAR(255) NOT NULL,
file_path VARCHAR(500) NOT NULL,
file_size BIGINT,
mime_type VARCHAR(100),
department VARCHAR(50),
archive_type VARCHAR(50),
retention_years INT DEFAULT 5,
upload_time DATETIME DEFAULT CURRENT_TIMESTAMP,
upload_user VARCHAR(50),
storage_key VARCHAR(255) UNIQUE,
INDEX idx_department (department),
INDEX idx_upload_time (upload_time)
);
CREATE TABLE access_log (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
archive_id BIGINT,
access_user VARCHAR(50),
access_time DATETIME DEFAULT CURRENT_TIMESTAMP,
operation VARCHAR(20),
FOREIGN KEY (archive_id) REFERENCES archive_metadata(id)
);

启动服务：

docker-compose up -d

三、核心功能实现

3.1 档案上传接口

创建Spring Boot控制器ArchiveController.java：

@RestController
@RequestMapping("/api/archive")
public class ArchiveController {
@Autowired
private MinioClient minioClient;
@PostMapping("/upload")
public ResponseEntity uploadArchive(
@RequestParam("file") MultipartFile file,
@RequestParam String department,
@RequestParam String archiveType,
@RequestParam Integer retentionYears) {
try {
// 1. 生成存储路径
String year = LocalDate.now().getYear() + "";
String objectName = year + "/" + department + "/"
+ archiveType + "/" + UUID.randomUUID() + "_"
+ file.getOriginalFilename();
// 2. 上传到MinIO
minioClient.putObject(
PutObjectArgs.builder()
.bucket("archives")
.object(objectName)
.stream(file.getInputStream(),
file.getSize(), -1)
.contentType(file.getContentType())
.build()
);
// 3. 保存元数据
ArchiveMetadata metadata = new ArchiveMetadata();
metadata.setFileName(file.getOriginalFilename());
metadata.setFilePath(objectName);
metadata.setFileSize(file.getSize());
metadata.setMimeType(file.getContentType());
metadata.setDepartment(department);
metadata.setArchiveType(archiveType);
metadata.setRetentionYears(retentionYears);
metadata.setStorageKey(objectName);
metadataMapper.insert(metadata);
// 4. 索引到Elasticsearch
IndexRequest request = new IndexRequest("archives")
.id(metadata.getId().toString())
.source(
"fileName", metadata.getFileName(),
"department", metadata.getDepartment(),
"archiveType", metadata.getArchiveType(),
"content", extractText(file), // OCR或文本提取
"uploadTime", metadata.getUploadTime()
);
elasticsearchClient.index(request, RequestOptions.DEFAULT);
return ResponseEntity.ok(
Map.of("id", metadata.getId(),
"message", "上传成功")
);
} catch (Exception e) {
return ResponseEntity.status(500)
.body(Map.of("error", e.getMessage()));
}
}
}

3.2 档案检索功能

创建检索服务SearchService.java：

@Service
public class SearchService {
public List searchArchives(
String keyword,
String department,
LocalDate startDate,
LocalDate endDate) {
SearchRequest searchRequest = new SearchRequest("archives");
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
if (StringUtils.hasText(keyword)) {
boolQuery.must(QueryBuilders.multiMatchQuery(keyword)
.field("fileName", 3.0f)
.field("content")
.type(MultiMatchQueryBuilder.Type.BEST_FIELDS));
}
if (StringUtils.hasText(department)) {
boolQuery.filter(QueryBuilders.termQuery("department", department));
}
if (startDate != null && endDate != null) {
boolQuery.filter(QueryBuilders.rangeQuery("uploadTime")
.gte(startDate.atStartOfDay())
.lte(endDate.atTime(LocalTime.MAX)));
}
sourceBuilder.query(boolQuery);
sourceBuilder.from(0);
sourceBuilder.size(100);
sourceBuilder.sort(new FieldSortBuilder("uploadTime").order(SortOrder.DESC));
searchRequest.source(sourceBuilder);
try {
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
return Arrays.stream(response.getHits().getHits())
.map(hit -> convertToArchiveVO(hit))
.collect(Collectors.toList());
} catch (IOException e) {
throw new RuntimeException("搜索失败", e);
}
}
}

四、生命周期管理

4.1 定期清理任务

创建Spring Boot定时任务CleanupTask.java：

@Component
public class CleanupTask {
@Scheduled(cron = "0 0 2   ?") // 每天凌晨2点执行
@Transactional
public void cleanupExpiredArchives() {
// 1. 查询过期档案
LocalDate cutoffDate = LocalDate.now().minusYears(10);
List expired = metadataMapper.selectExpired(cutoffDate);
for (ArchiveMetadata archive : expired) {
try {
// 2. 从MinIO删除文件
minioClient.removeObject(
RemoveObjectArgs.builder()
.bucket("archives")
.object(archive.getStorageKey())
.build()
);
// 3. 从Elasticsearch删除索引
DeleteRequest deleteRequest = new DeleteRequest("archives",
archive.getId().toString());
elasticsearchClient.delete(deleteRequest, RequestOptions.DEFAULT);
// 4. 从数据库删除元数据
metadataMapper.deleteById(archive.getId());
// 5. 记录删除日志
AccessLog log = new AccessLog();
log.setArchiveId(archive.getId());
log.setOperation("AUTO_DELETE");
log.setAccessUser("system");
accessLogMapper.insert(log);
} catch (Exception e) {
log.error("删除档案失败: {}", archive.getId(), e);
}
}
}
}

4.2 完整性校验

创建校验任务ValidationTask.java：

@Scheduled(cron = "0 0 4   ?") // 每天凌晨4点执行
public void validateArchives() {
List allArchives = metadataMapper.selectList(null);
for (ArchiveMetadata archive : allArchives) {
try {
// 检查MinIO文件是否存在
StatObjectResponse stat = minioClient.statObject(
StatObjectArgs.builder()
.bucket("archives")
.object(archive.getStorageKey())
.build()
);
// 验证文件大小是否匹配
if (stat.size() != archive.getFileSize()) {
log.warn("档案大小不匹配: {}", archive.getId());
sendAlert("档案大小异常", archive);
}
} catch (ErrorResponseException e) {
if (e.errorResponse().code().equals("NoSuchKey")) {
log.error("档案文件丢失: {}", archive.getId());
sendAlert("档案文件丢失", archive);
}
}
}
}

五、监控与维护

5.1 健康检查接口

@GetMapping("/health")
public Map healthCheck() {
Map health = new HashMap<>();
// 检查MySQL
try {
dataSource.getConnection().close();
health.put("mysql", "UP");
} catch (Exception e) {
health.put("mysql", "DOWN");
}
// 检查MinIO
try {
minioClient.listBuckets();
health.put("minio", "UP");
} catch (Exception e) {
health.put("minio", "DOWN");
}
// 检查Elasticsearch
try {
elasticsearchClient.ping(RequestOptions.DEFAULT);
health.put("elasticsearch", "UP");
} catch (Exception e) {
health.put("elasticsearch", "DOWN");
}
return health;
}

5.2 关键指标监控

在服务器上创建监控脚本monitor.sh：

!/bin/bash
磁盘使用率
DISK_USAGE=$(df -h /data | awk 'NR==2 {print $5}' | sed 's/%//')
服务状态
MYSQL_STATUS=$(systemctl is-active docker.mysql)
MINIO_STATUS=$(systemctl is-active docker.minio)
档案数量统计
TOTAL_ARCHIVES=$(mysql -u root -pRootPass123! -e "SELECT COUNT() FROM archive_metadata" -N)
输出JSON格式
echo "{
\"timestamp\": \"$(date -Iseconds)\",
\"disk_usage\": $DISK_USAGE,
\"mysql_status\": \"$MYSQL_STATUS\",
\"minio_status\": \"$MINIO_STATUS\",
\"total_archives\": $TOTAL_ARCHIVES
}" > /var/log/archive_monitor.json

设置定时任务：

 编辑crontab
crontab -e
添加以下行（每5分钟执行一次）
/5     /opt/archive-system/scripts/monitor.sh

六、备份与恢复

6.1 数据备份策略

创建备份脚本backup.sh：

!/bin/bash
BACKUP_DIR="/backup/archive-system"
DATE=$(date +%Y%m%d_%H%M%S)
备份MySQL
mysqldump -u root -pRootPass123! archive_db > \
$BACKUP_DIR/mysql_$DATE.sql
备份MinIO元数据（使用mc命令）
mc mirror --overwrite local/archives \
$BACKUP_DIR/minio_$DATE/
备份Elasticsearch索引
curl -X GET "localhost:9200/_snapshot/backup_repo/snapshot_$DATE?wait_for_completion=true"
保留最近7天备份
find $BACKUP_DIR -name ".sql" -mtime +7 -delete
find $BACKUP_DIR -name "minio_" -type d -mtime +7 | xargs rm -rf

6.2 灾难恢复流程

恢复步骤：

1. 停止所有服务： docker-compose down
2. 恢复MySQL： mysql -u root -pRootPass123! archive_db < backup.sql
3. 恢复MinIO数据： mc mirror --overwrite backup/minio_date/ local/archives
4. 恢复Elasticsearch：

   curl -X POST "localhost:9200/_snapshot/backup_repo/snapshot_date/_restore"
   

5. 启动服务： docker-compose up -d
6. 验证数据完整性： curl http://localhost:8080/api/archive/health

按照以上步骤完整实施，即可获得一个功能完备、可维护的档案短期保存系统。所有代码和配置均为生产可用版本，可根据实际需求调整参数。