面试题：Java中StringTokenizer在复杂文本处理及状态维护场景下的应用

Question

给定一个文本字符串，其中包含不同语言的代码片段，用'---'分隔不同代码片段，每个代码片段内部又用';'分隔不同语句。例如：'System.out.println("Hello");int num = 10;---print("World");x = 20;---cout << "Hi";int a = 5;'。请使用StringTokenizer，将每个代码片段及其内部的语句分别提取出来，并统计每种语言代码片段的数量（假设仅识别Java、Python、C++这三种语言的代码片段，可通过典型关键字识别，如Java的System，Python的print，C++的cout），编写Java代码实现该功能。

Answer 1

import java.util.StringTokenizer;
import java.util.HashMap;
import java.util.Map;

public class CodeFragmentAnalyzer {
    public static void main(String[] args) {
        String text = "System.out.println(\"Hello\");int num = 10;---print(\"World\");x = 20;---cout << \"Hi\";int a = 5;";
        StringTokenizer outerTokenizer = new StringTokenizer(text, "---");
        Map<String, Integer> languageCountMap = new HashMap<>();
        languageCountMap.put("Java", 0);
        languageCountMap.put("Python", 0);
        languageCountMap.put("C++", 0);

        while (outerTokenizer.hasMoreTokens()) {
            String codeFragment = outerTokenizer.nextToken();
            StringTokenizer innerTokenizer = new StringTokenizer(codeFragment, ";");
            String language = identifyLanguage(codeFragment);
            if (language != null) {
                languageCountMap.put(language, languageCountMap.get(language) + 1);
            }
            System.out.println("Code Fragment: " + codeFragment);
            while (innerTokenizer.hasMoreTokens()) {
                System.out.println("Statement: " + innerTokenizer.nextToken());
            }
        }

        for (Map.Entry<String, Integer> entry : languageCountMap.entrySet()) {
            System.out.println(entry.getKey() + " code fragments count: " + entry.getValue());
        }
    }

    private static String identifyLanguage(String codeFragment) {
        if (codeFragment.contains("System")) {
            return "Java";
        } else if (codeFragment.contains("print")) {
            return "Python";
        } else if (codeFragment.contains("cout")) {
            return "C++";
        }
        return null;
    }
}