Skip to content

[Feat] 자기소개서 분석 Hybrid Exact 기준 구현 #174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
whc9999 merged 2 commits into from
Jul 25, 2026
Merged

[Feat] 자기소개서 분석 Hybrid Exact 기준 구현 #174

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
91c1a8a
[Feat] 자기소개서 분석 Hybrid Exact 기준 구현
whc9999 Jul 25, 2026
ecfe9f4
[Fix] Hybrid Exact 리뷰 유효 항목 반영
whc9999 Jul 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions evaluation/evaluation_nlg_judge_hybrid_comparison.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
sourceResultFile,caseCount,successCount,averageRelevance,averageProblemValidity,averageSentenceTypeConsistency,averageReasonCorrectness,averageContextAwareness,averageFaithfulness,averageTenseConsistency,averageUsability,averageNonMeta,averageMeaningPreservation,noAnalysisAppropriateness,strengthsPrecision,strengthsCoverage,missingKeywordsPrecision,missingKeywordsCoverage,overallUsefulness,averageJudgeInputTokens,averageJudgeOutputTokens,averageJudgeLatencyMs,averageAnalysisCount,metaImprovementRate,unsupportedFactRate,falsePositiveAnalysisRate,fatalErrorRate,errorCodeCounts
evaluation/evaluation_nlg_judge_v5a_v2.csv,20,20,3.06,2.75,3.75,3.19,2.06,4.0,4.13,3.81,3.69,4.06,2.05,4.45,4.45,3.45,3.45,2.75,,,4214.25,0.8,0.0,0.0,0.0,0.0,"{""CONTEXT_IGNORED"":4,""MISSED_ANALYSIS"":18,""MISSED_MISSING_KEYWORD"":10,""NONE"":1}"
evaluation/evaluation_nlg_judge_two_pass_v2_1_v2.csv,20,20,3.33,2.5,3.5,3.33,2.17,4.0,4.17,3.5,3.33,4.0,1.85,4.14,4.14,4.07,4.07,2.6,,,3821.65,0.45,0.0,0.0,0.0,0.0,"{""CONTEXT_IGNORED"":2,""MISSED_ANALYSIS"":19,""MISSED_MISSING_KEYWORD"":2,""MISSED_STRENGTH"":2,""NONE"":1}"
evaluation/evaluation_nlg_judge_hybrid_v1.csv,20,20,3.17,2.75,3.58,3.17,2.0,3.92,4.08,3.67,4.0,4.08,2.0,3.55,3.55,1.95,1.95,2.1,3617.55,235.75,3506.55,0.65,0.0,0.0,0.0,0.0,"{""CONTEXT_IGNORED"":1,""MISSED_ANALYSIS"":20,""MISSED_MISSING_KEYWORD"":20}"
21 changes: 21 additions & 0 deletions evaluation/evaluation_nlg_judge_hybrid_v1.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
caseId,sourceResultFile,analysisCount,averageRelevance,averageProblemValidity,averageSentenceTypeConsistency,averageReasonCorrectness,averageContextAwareness,averageFaithfulness,averageTenseConsistency,averageUsability,averageNonMeta,averageMeaningPreservation,noAnalysisAppropriateness,strengthsPrecision,strengthsCoverage,missingKeywordsPrecision,missingKeywordsCoverage,overallUsefulness,errorCodes,shortRationale,judgeInputTokens,judgeOutputTokens,judgeLatencyMs,failureStage
EV-01,evaluation/evaluation_ai_results_hybrid_v1.csv,0,,,,,,,,,,,2,4,4,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","답변에 명백한 문제 문장이 없으나, 분석이 비어 있어 중요한 첨삭 대상을 놓쳤습니다. 또한, JD의 핵심 경험 요구사항이 누락되었습니다.",3642,153,3460,
EV-02,evaluation/evaluation_ai_results_hybrid_v1.csv,1,3.0,3.0,4.0,3.0,2.0,4.0,4.0,4.0,4.0,4.0,2,4,4,1,1,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원자의 경험이 잘 드러나지 않은 부분이 있으며, JD의 핵심 키워드가 누락되었습니다. 전반적으로 개선이 필요합니다.",2711,289,3483,
EV-03,evaluation/evaluation_ai_results_hybrid_v1.csv,1,3.0,3.0,4.0,3.0,2.0,4.0,4.0,3.0,4.0,4.0,2,4,4,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","분석에서 중요한 경험과 강점이 누락되었으며, 명백한 문제 문장이 존재하지만 분석이 부족하여 전반적인 유용성이 낮게 평가되었습니다.",4208,287,4661,
EV-04,evaluation/evaluation_ai_results_hybrid_v1.csv,0,,,,,,,,,,,2,3,3,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","분석이 전혀 이루어지지 않았고, 중요한 첨삭 대상을 놓쳤습니다. 또한, JD의 핵심 경험 요구사항이 누락되었습니다.",4793,147,2841,
EV-05,evaluation/evaluation_ai_results_hybrid_v1.csv,1,4.0,3.0,4.0,3.0,2.0,5.0,5.0,4.0,5.0,5.0,2,3,3,2,2,3,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원자의 포부는 명확하나 구체적인 실행 방법이 부족하며, JD의 핵심 경험 요구사항이 누락되었습니다. 전반적으로 유용성은 중간 수준입니다.",3120,301,3162,
EV-06,evaluation/evaluation_ai_results_hybrid_v1.csv,1,3.0,3.0,4.0,3.0,2.0,4.0,4.0,3.0,4.0,4.0,2,3,3,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","자기소개서에서 주요 경험과 강점이 잘 드러나지 않았고, 중요한 키워드가 누락되었습니다. 분석이 부족하여 전반적인 유용성이 낮게 평가되었습니다.",5796,290,3698,
EV-07,evaluation/evaluation_ai_results_hybrid_v1.csv,0,,,,,,,,,,,2,5,5,1,1,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","분석이 전혀 이루어지지 않았고, 중요한 경험 키워드가 누락되었습니다. 전반적으로 개선이 필요합니다.",2545,139,1994,
EV-08,evaluation/evaluation_ai_results_hybrid_v1.csv,0,,,,,,,,,,,2,3,3,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원자의 경험이 JD의 주요 요구사항을 충분히 반영하지 못하고 있으며, 분석이 전혀 이루어지지 않아 중요한 문제를 놓쳤습니다.",4207,144,2814,
EV-09,evaluation/evaluation_ai_results_hybrid_v1.csv,1,3.0,3.0,4.0,3.0,2.0,4.0,4.0,4.0,4.0,4.0,2,4,4,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원자의 경험은 유용하지만, JD의 주요 요구사항인 부가세 및 법인세 신고 실무 경험과 결산 및 관리회계 실무 경험이 누락되었습니다. 또한, 분석이 부족하여 중요한 문제를 놓쳤습니다.",3584,324,4247,
EV-10,evaluation/evaluation_ai_results_hybrid_v1.csv,1,3.0,3.0,3.0,3.0,2.0,4.0,4.0,4.0,4.0,4.0,2,4,4,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원서에서 주요 업무와 관련된 경험이 부족하며, 분석이 결여된 부분이 있어 전반적인 유용성이 낮습니다.",3521,304,4867,
EV-11,evaluation/evaluation_ai_results_hybrid_v1.csv,1,4.0,3.0,4.0,3.0,2.0,4.0,4.0,4.0,4.0,4.0,2,4,4,2,2,3,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원 동기와 포부는 잘 드러나지만, 구체적인 실행 방법과 JD의 핵심 경험 요구사항이 누락되어 있습니다.",3439,299,6653,
EV-12,evaluation/evaluation_ai_results_hybrid_v1.csv,1,3.0,2.0,3.0,3.0,2.0,3.0,4.0,3.0,4.0,4.0,2,4,4,2,2,2,"[""CONTEXT_IGNORED"",""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","분석에서 중요한 경험 키워드가 누락되었고, 문맥을 무시한 평가가 있었습니다. 전반적으로 개선이 필요합니다.",3806,295,3174,
EV-13,evaluation/evaluation_ai_results_hybrid_v1.csv,1,3.0,3.0,3.0,3.0,2.0,4.0,4.0,4.0,4.0,4.0,2,4,4,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원자의 경험이 직무와 관련된 부분이 부족하며, 주요 키워드가 누락되었습니다. 분석이 부족하여 중요한 문제를 놓쳤습니다.",3511,270,3689,
EV-14,evaluation/evaluation_ai_results_hybrid_v1.csv,0,,,,,,,,,,,2,3,3,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원자의 경험이 JD의 주요 요구사항을 충분히 반영하지 못하고 있으며, 분석이 전혀 이루어지지 않아 중요한 첨삭 대상을 놓쳤습니다.",3764,147,1857,
EV-15,evaluation/evaluation_ai_results_hybrid_v1.csv,0,,,,,,,,,,,2,4,4,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원자의 경험과 강점이 잘 드러나지 않았고, 주요 요구사항이 누락되었습니다. 분석이 전혀 없어서 중요한 문제를 놓쳤습니다.",2927,145,2015,
EV-16,evaluation/evaluation_ai_results_hybrid_v1.csv,1,3.0,3.0,3.0,4.0,2.0,4.0,4.0,4.0,4.0,4.0,2,2,2,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","분석이 부족하여 중요한 문제를 놓쳤고, JD의 핵심 경험 요구사항이 누락되었습니다. 전반적으로 개선이 필요합니다.",3929,293,4032,
EV-17,evaluation/evaluation_ai_results_hybrid_v1.csv,1,3.0,2.0,3.0,3.0,2.0,3.0,4.0,3.0,3.0,4.0,2,4,4,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]",분석이 부족하고 주요 경험 키워드가 누락되어 전반적인 유용성이 낮습니다.,3064,270,3348,
EV-18,evaluation/evaluation_ai_results_hybrid_v1.csv,1,3.0,2.0,4.0,4.0,2.0,4.0,4.0,4.0,4.0,4.0,2,3,3,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","분석에서 병원 행정업무 지원 관련 경험이 부족하다는 점이 명확히 드러났으나, 이에 대한 구체적인 개선이 제시되지 않아 전반적인 유용성이 낮게 평가되었습니다.",3142,281,4608,
EV-19,evaluation/evaluation_ai_results_hybrid_v1.csv,0,,,,,,,,,,,2,4,4,2,2,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원자의 경험과 동기는 명확하나, JD의 주요 요구사항을 충분히 반영하지 못하고 있으며, 분석이 전혀 이루어지지 않아 중요한 첨삭 대상을 놓쳤습니다.",3079,154,2241,
EV-20,evaluation/evaluation_ai_results_hybrid_v1.csv,1,,,,,,,,,,,2,2,2,3,3,2,"[""MISSED_ANALYSIS"",""MISSED_MISSING_KEYWORD""]","지원자의 포부는 있으나 구체적인 실행 방법이 부족하며, JD의 주요 경험 요구사항이 누락되었습니다. 전반적으로 개선이 필요합니다.",3563,183,3287,
7 changes: 5 additions & 2 deletions src/main/java/com/jobdri/jobdri_api/domain/analysis/evaluation/EvaluationAnalysisRunner.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ public class EvaluationAnalysisRunner implements ApplicationRunner {
@Value("${evaluation.nlg-judge.enabled:false}")
private boolean nlgJudgeEnabled;

@Value("${evaluation.hybrid-merge.enabled:false}")
private boolean hybridMergeEnabled;

private final EvaluationAnalysisBatchService evaluationAnalysisBatchService;
private final EvaluationExitCoordinator evaluationExitCoordinator;
private final Environment environment;
Expand Down Expand Up @@ -88,9 +91,9 @@ void validateProfiles() {
}

void validateExecutionProperties() {
if (nlgJudgeEnabled) {
if (nlgJudgeEnabled || hybridMergeEnabled) {
throw new IllegalArgumentException(
"evaluation.analysis.enabled와 evaluation.nlg-judge.enabled를 동시에 true로 설정할 수 없습니다."
"evaluation.analysis.enabled는 evaluation.nlg-judge.enabled 또는 evaluation.hybrid-merge.enabled와 동시에 true로 설정할 수 없습니다."
);
}
if (!StringUtils.hasText(inputPath)) {
Expand Down
131 changes: 77 additions & 54 deletions src/main/java/com/jobdri/jobdri_api/domain/analysis/evaluation/EvaluationCsvSupport.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,60 +63,53 @@ private static CsvRows readRows(Path path) throws IOException {
}

static void write(Path path, List<EvaluationAnalysisResult> results) throws IOException {
Path parent = path.toAbsolutePath().getParent();
if (parent != null) {
Files.createDirectories(parent);
}

try (BufferedWriter writer = Files.newBufferedWriter(path, StandardCharsets.UTF_8)) {
writeRow(writer, List.of(
"caseId",
"jobCategoryMiddle",
"jobCategorySmall",
"mainTasks",
"qualifications",
"preferences",
"question",
"answer",
"aiScore",
"aiJobFit",
"aiImpact",
"aiCompleteness",
"aiFeedback",
"aiMissingKeywordsJson",
"aiQuestionAnalysesJson",
"rawLlmResponseJson",
"rawCandidateResponseJson",
"sanitizedCandidateResponseJson",
"candidateReviewResponseJson",
"candidateCount",
"candidateAnalysisCount",
"candidateStrengthCount",
"candidateMissingKeywordCount",
"acceptedCandidateCount",
"rejectedCandidateCount",
"rejectionCodeCounts",
"finalAnalysisCount",
"strengthCandidateCount",
"finalStrengthCount",
"missingKeywordCandidateCount",
"finalMissingKeywordCount",
"candidateCallLatencyMs",
"finalCallLatencyMs",
"candidateLatencyMs",
"finalLatencyMs",
"candidateInputTokens",
"candidateOutputTokens",
"finalInputTokens",
"finalOutputTokens",
"totalInputTokens",
"totalOutputTokens",
"failureStage",
"errorMessage",
"createdAt"
));

for (EvaluationAnalysisResult result : results) {
writeCsv(path, List.of(
"caseId",
"jobCategoryMiddle",
"jobCategorySmall",
"mainTasks",
"qualifications",
"preferences",
"question",
"answer",
"aiScore",
"aiJobFit",
"aiImpact",
"aiCompleteness",
"aiFeedback",
"aiMissingKeywordsJson",
"aiQuestionAnalysesJson",
"rawLlmResponseJson",
"rawCandidateResponseJson",
"sanitizedCandidateResponseJson",
"candidateReviewResponseJson",
"candidateCount",
"candidateAnalysisCount",
"candidateStrengthCount",
"candidateMissingKeywordCount",
"acceptedCandidateCount",
"rejectedCandidateCount",
"rejectionCodeCounts",
"finalAnalysisCount",
"strengthCandidateCount",
"finalStrengthCount",
"missingKeywordCandidateCount",
"finalMissingKeywordCount",
"candidateCallLatencyMs",
"finalCallLatencyMs",
"candidateLatencyMs",
"finalLatencyMs",
"candidateInputTokens",
"candidateOutputTokens",
"finalInputTokens",
"finalOutputTokens",
"totalInputTokens",
"totalOutputTokens",
"failureStage",
"errorMessage",
"createdAt"
), writer -> {
for (EvaluationAnalysisResult result : results == null ? List.<EvaluationAnalysisResult>of() : results) {
writeRow(writer, List.of(
value(result.caseId()),
value(result.jobCategoryMiddle()),
Expand Down Expand Up @@ -164,7 +157,37 @@ static void write(Path path, List<EvaluationAnalysisResult> results) throws IOEx
value(result.createdAt())
));
}
});
}

static void writeRows(Path path, List<String> headers, List<Map<String, String>> rows) throws IOException {
List<String> safeHeaders = headers == null ? List.of() : headers;
writeCsv(path, safeHeaders, writer -> {
for (Map<String, String> row : rows == null ? List.<Map<String, String>>of() : rows) {
List<String> values = new ArrayList<>();
for (String header : safeHeaders) {
values.add(value(row.get(header)));
}
writeRow(writer, values);
}
});
}

private static void writeCsv(Path path, List<String> headers, CsvRowsWriter rowsWriter) throws IOException {
Path parent = path.toAbsolutePath().getParent();
if (parent != null) {
Files.createDirectories(parent);
}

try (BufferedWriter writer = Files.newBufferedWriter(path, StandardCharsets.UTF_8)) {
writeRow(writer, headers == null ? List.of() : headers);
rowsWriter.write(writer);
}
}

Comment thread
coderabbitai[bot] marked this conversation as resolved.
@FunctionalInterface
private interface CsvRowsWriter {
void write(BufferedWriter writer) throws IOException;
}

private static List<List<String>> parseRows(String content) {
Expand Down
11 changes: 9 additions & 2 deletions .../java/com/jobdri/jobdri_api/domain/analysis/evaluation/EvaluationRunnerFlagValidator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,18 @@ class EvaluationRunnerFlagValidator implements SmartInitializingSingleton {
@Value("${evaluation.nlg-judge.enabled:false}")
private boolean nlgJudgeEnabled;

@Value("${evaluation.hybrid-merge.enabled:false}")
private boolean hybridMergeEnabled;

@Override
public void afterSingletonsInstantiated() {
if (analysisEvaluationEnabled && nlgJudgeEnabled) {
int enabledRunnerCount = 0;
enabledRunnerCount += analysisEvaluationEnabled ? 1 : 0;
enabledRunnerCount += nlgJudgeEnabled ? 1 : 0;
enabledRunnerCount += hybridMergeEnabled ? 1 : 0;
if (enabledRunnerCount > 1) {
throw new IllegalStateException(
"evaluation.analysis.enabled와 evaluation.nlg-judge.enabled를 동시에 true로 설정할 수 없습니다."
"analysis-eval runner flags are mutually exclusive: evaluation.analysis.enabled, evaluation.nlg-judge.enabled, evaluation.hybrid-merge.enabled"
);
}
}
Expand Down
Loading
Loading