首页
社区
课程
招聘
问答 CTF 排行榜 知识库 工具下载 看雪峰会 看雪商城 证书查询
  1. 看雪社区
  2. 智能设备
    3. 发新帖
[原创]EMBA安装、使用与源代码分析_0
2023-5-1 15:56 21322

[原创]EMBA安装、使用与源代码分析_0

AE86 活跃值
2023-5-1 15:56
21322

项目地址：https://github.com/e-m-b-a/emba
本次分析对象的最后一次提交：https://github.com/e-m-b-a/emba/commit/de9dc53980962c8a62ea1239c9039e87962b32e5
克隆项目

1
git clone https://github.com/e-m-b-a/emba.git

介绍

EMBA 被设计为渗透测试人员的中央固件分析工具。它支持完整的安全分析过程，从固件提取过程开始，通过仿真进行静态分析和动态分析，最后生成 Web 报告。EMBA 自动发现固件中可能存在的弱点和漏洞。例如不安全的二进制文件、旧的和过时的软件组件、可能易受攻击的脚本或硬编码密码。EMBA是一个命令行工具，可以选择生成易于使用的 Web 报告以供进一步分析。
EMBA 结合了多种成熟的分析工具，可以通过一个简单的命令启动。之后，它会测试固件是否存在可能的安全风险和感兴趣的区域以进行进一步调查。无需手动安装所有帮助程序，执行集成安装脚本后，您就可以测试固件了。
EMBA 旨在协助渗透测试人员，而不是作为一个没有人工交互的独立工具。EMBA 应该提供尽可能多的关于固件的信息，测试人员可以决定重点领域，并负责验证和解释结果。
官方视频：
https://youtu.be/_dvdy3klFFY
项目采用纯Shell编写，非常适合这种需要结合许多外部工具并执行大量命令的工具。
项目统计：
Summary
Date : 2023-04-19 10:38:56
Total : 225 files, 28187 codes, 3873 comments, 4716 blanks, all 36776 lines
Languages

language files code comment blank total
Shell Script 173 24,559 3,657 4,361 32,577
Properties 22 1,269 70 20 1,359
XML 2 1,034 1 3 1,038
YAML 17 555 92 86 733
CSS 1 343 14 77 434
Markdown 8 309 14 146 469
HTML 1 106 22 15 143
Docker 1 12 3 8 23
 

Directories

path files code comment blank total
. 225 28,187 3,873 4,716 36,776
. (Files) 9 812 87 220 1,119
.github 19 549 81 116 746
.github (Files) 2 6 1 15 22
.github\ISSUE_TEMPLATE 2 47 0 19 66
.github\workflows 15 496 80 82 658
config 62 1,395 110 101 1,606
config (Files) 22 1,269 70 20 1,359
config\report_templates 40 126 40 81 247
helpers 26 5,523 729 672 6,924
installer 26 1,734 519 361 2,614
modules 83 18,174 2,347 3,246 23,767
modules (Files) 77 17,704 2,269 3,162 23,135
modules\L10_system_emulation 6 470 78 84 632

由代码量可知其主要功能集中在modules文件夹内。通过后面的分析可以知道，EMBA将根据运行参数分别调用各个模块对固件进行静态或动态的分析。

 

项目结构

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
> tree .                
.
├── check_project.sh
├── CODE_OF_CONDUCT.md
├── config
│   ├── banner
│   │   ├── ...
│   │   └── Vegas_Edt-v1.1.0.txt
│   ├── report_templates
│   │   ├── ...
│   │   └── S99_grepit-pre.sh
│   ├── ...
│   └── trickest_cve-db.txt
├── CONTRIBUTING.md
├── CONTRIBUTORS.md
├── docker-compose.yml
├── Dockerfile
├── emba
├── helpers
│   ├── ...
│   └── trickest_db_update.sh
├── installer
│   ├── ...
│   └── wickStrictModeFail.sh
├── installer.sh
├── LICENSE
├── licenses
│   └── LICENSE-MIT-firmae-firmadyne.txt
├── modules
│   ├── L10_system_emulation
│   │   ├── ...
│   │   └── run_service.sh
│   ├── ...
│   └── template_module.sh
├── README.md
├── scan-profiles
│   ├── ...
│   └── full-scan.emba
└── SECURITY.md
 
9 directories, 250 files

描述：

  1. check_project.sh：用shellchecker检查./helpers、./modules、emba和自身中的所有shell脚本
  2. config/banner/*：banner的配置文件，用于输出横幅
  3. config/report_templates/*：一些模板输出的脚本，比如打印该文件是什么组件等。
  4. config/*：各种配置文件。包括用于各种特征匹配的正则库：开源协议版本库、证书、启动文件、命令执行、字符串相关操作敏感函数、配置文件、密钥文件、发行版检测、某些摄像头密钥文件、历史文件、http程序文件、网络服务有关文件、编译工具有关文件、已知漏洞&cve&linux、密码文件等等等等。
  5. helpers/*：提供一些有用的脚本，包括修复权限与链接、修复模拟、检查依赖、html生成等脚本。
  6. installer/*：安装有关的脚本。包括安装需要的工具等。
  7. modules/*：各种分析模块，包括二进制检查、vmdk提取、nmap扫描、开源协议摘要等等。
  8. scan-profiles/*：存放了一些EMBA profile，用与配置具体的分析任务。

安装

建议：

  1. 使用最新最新的Kali Linux虚拟机（Ubuntu也可以）
  2. 使用默认docker模式
  3. 如果你不清楚自己在干啥，请不要在开发者模式（-D）下使用EMBA，因为某些检测模块会执行恶意代码从而损坏宿主机。
  4. 经典模式将使用Docker运行。你只需要安装Docker和cve-search。
  5. 只能使用x86-64架构的虚拟机运行
  6. 最低8GB内存，4核CPU，30G以上磁盘空间（若内存低于8G安装 cve-search 时会卡死，并且固件的深度提取会使用大量磁盘空间！）
  7. docker-compose 1.29

典型安装

1
sudo ./installer.sh -d

注意！
国内用户记得配置代理，不然git啥的会报错。另外，sudo运行的命令不会继承当前用户的环境变量。
cve-search 的安装需要配置代理，需要修改安装流程（不然特别特别慢，还容易失败）

1
2
3
4
5
6
7
8
9
10
11
# installer\IF20_cve_search.sh line 38
      # we always need the cve-search stuff:
      if ! [[ -d external/cve-search ]]; then
        git clone https://github.com/EMBA-support-repos/cve-search.git external/cve-search
 
# external\cve-search\lib\Config.py line 37
    default = {
        "redisHost": "localhost",
        "redisPort": 6379,
        "redisQ": 9,
        ...

这将安装所需要的依赖工具，例如cve-search、下载大约6GB的docker镜像等等。

手动构建：
docker-compose build emba

 

安装完后测试 cve-search：./external/cve-search/bin/search.py -p busybox
图片描述
默认方式安装和运行操作都将使用 docker-compose 运行 emba

分析安装过程

首先加载helpers或installer文件夹下的helpers_emba_load_strict_settings.sh，并执行其中的load_strict_mode_settings函数用于加载严格模式的设置
安装docker
加载所有installer文件夹下的*.sh（但是一般都是以函数的形式存在，加载时并不会安装）
检查参数个数，必须为1
检查参数cCdDFghlr，设置对应的安装标记，然后根据安装标记进行针对性安装。在此过程还会检查wsl等环境进行针对性安装。
cve-search is always installed on the host，使用IF20_cve_search函数。

cve-search使用mongodb存储，并会使用pip安装redis等组件。

使用

静态分析

必须指定日志和固件文件：

1
sudo ./emba.sh -l ./log -f ./firmware
  • 请使用 sudo 运行
  • 日志文件会很大
  • 目前支持的架构：MIPS、ARM、PPC、x86 和 x64
  • 你也可以使用写好的扫描配置文件：scan-profiles/*.emba

检查内核配置

使用 checksec.sh 的kernel checker仅测试内核配置：

1
sudo ./emba.sh -l ./logs/kernel_conf -k ./kernel.config

如果添加-f ./firmware，它将忽略-k并在固件中搜索内核配置
checksec相信打过 pwn 的人应该都很熟悉。关于 checksec 的 kernel checker：
checksec 是一种用于检查Linux系统安全性的工具，其中包括一个kernel checker。checksec kernel checker通过检查当前正在运行的内核的各种安全配置选项来帮助用户评估其安全性。
checksec kernel checker可以检查以下配置选项：

  1. ASLR（address space layout randomization）：用于随机化进程虚拟地址空间布局，以减少攻击者利用内存漏洞的可能性。
  2. NX（no execute）：用于禁止在可执行内存区域中执行代码，以防止攻击者利用缓冲区溢出等漏洞注入和执行恶意代码。
  3. KASLR（kernel address space layout randomization）：用于随机化内核代码和数据的物理地址，以增加攻击者突破内核安全的难度。
  4. Stack Protector：用于在栈上保护函数返回地址，以防止栈溢出攻击。

通过检查这些配置选项，checksec kernel checker可以提供有关系统安全性的重要信息，并帮助用户确定是否需要采取额外的安全措施。

主文件（emba）代码分析

运行主文件。首先定义了一些函数，然后在最后调用主函数，并传递所有参数：
图片描述
然后解析参数、检查参数、文件、路径、依赖工具等。设置相关环境变量。
监控docker日志和通知：
图片描述
helpers\helpers_emba_print.sh：
图片描述

如果指定了-t参数，则会设置THREADED和MAX_MOD_THREADS参数（MAX_MOD_THREADS为核心数乘2)：

1
2
3
4
5
6
# calculate the maximum threads per module
if [[ ${THREADED} -eq 1 ]] && [[ "${MAX_MOD_THREADS}" -eq 0 ]]; then
  # the maximum threads per modules - if this value does not match adjust it via
  # local MAX_MOD_THREADS=123 in module area
  export MAX_MOD_THREADS="$(( 2* "$(grep -c ^processor /proc/cpuinfo)" ))"
fi

预处理HTML输出

1
2
3
4
5
6
7
8
9
# Change log output to color for web report and prepare report
if [[ ${HTML} -eq 1 ]] ; then
  if [[ ${FORMAT_LOG} -eq 0 ]] ; then
    FORMAT_LOG=1
    print_output "[*] Activate colored log for webreport" "no_log"
  fi
  print_output "[*] Prepare webreport" "no_log"
  prepare_report
fi

带-k参数运行：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#######################################################################################
# Kernel configuration check
#######################################################################################
if [[ "${KERNEL}" -eq 1 ]]; then
  if [[ ${IN_DOCKER} -eq 1 ]] && [[ -f "${LOG_DIR}"/kernel_config ]]; then
    export KERNEL_CONFIG="${LOG_DIR}"/kernel_config
  fi
 
  if ! [[ -f "${KERNEL_CONFIG}" ]] ; then
    print_output "[-] Invalid kernel configuration file: ${ORANGE}${KERNEL_CONFIG}${NC}" "no_log"
    exit 1
  else
    if [[ ${IN_DOCKER} -eq 0 ]] ; then
      # we copy the kernel config file from outside the container into our log directory
      # further modules are using LOG_DIR/kernel_config for accessing the kernel config
      if [[ -d "${LOG_DIR}" ]] ; then
        cp "${KERNEL_CONFIG}" "${LOG_DIR}"/kernel_config
      else
        print_output "[!] Missing log directory" "no_log"
        exit 1
      fi
    fi
  fi
fi

会创建一个子进程，用来监控剩余空间容量：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
disk_space_monitor() {
  local DDISK="$LOG_DIR"
 
  while ! [[ -f "$MAIN_LOG" ]]; do
    sleep 1
  done
 
  while true; do
    # print_output "[*] Disk space monitoring active" "no_log"
    FREE_SPACE=$(df --output=avail "$DDISK" | awk 'NR==2')
    if [[ "$FREE_SPACE" -lt 10000000 ]]; then
      print_ln "no_log"
      print_output "[!] WARNING: EMBA is running out of disk space!" "main"
      print_output "[!] WARNING: EMBA is stopping now" "main"
      df -h || true
      print_ln "no_log"
      # give the container some more seconds for the cleanup process
      [[ "$IN_DOCKER" -eq 0 ]] && sleep 5
      cleaner 1
    fi
 
    if [[ -f "$MAIN_LOG" ]]; then
      if grep -q "Test ended\|EMBA failed" "$MAIN_LOG" 2>/dev/null; then
        break
      fi
    fi
 
    sleep 5
  done
}

运行容器，并执行分析：
图片描述
初始化状态栏：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
initial_status_bar() {
  # PID for box updater threads
  export PID_SYSTEM_LOAD=""
  export PID_STATUS=""
  export PID_MODULES=""
  export PID_STATUS_2=""
  # Path to status tmp file
  # each line is dedicated to a specific function
  # 1: Count of boxes visible
  # 2: CPU load string (needs to be cached, because it takes about a second to get the information)
  # 3: Start time of status bar - not exactly the same as the EMBA timer, but near enough to get an idea
  # 4: Count modules
  export STATUS_TMP_PATH=""
 
  # overwrites $LINES and "$COLUMNS" with the actual values of the window
  shopt -s checkwinsize; (:;:)
  local LINE_POS="$(( LINES - 6 ))"
  printf "\e[%s;1f\e[0J\e[%s;1f" "$LINE_POS" "$LINE_POS"
  reset
 
  # create new tmp file with empty lines
  STATUS_TMP_PATH="$TMP_DIR/status"
  if [[ ! -f "$STATUS_TMP_PATH" && -d "$TMP_DIR" ]] ; then
    echo -e "\\n\\n\\n\\n" > "$STATUS_TMP_PATH"
  fi
  # calculate boxes fitting and draw them
  local INITIAL_STR=""
  INITIAL_STR="\e[${LINE_POS};1f\e[0J\e[0;${LINE_POS}r\e[${LINE_POS};1f"
  if [[ $LINES -gt 10 ]] ; then
    # column has to be increased with 2 characters because of possible arrow column
    local ARROW_POS=0
    STATUS_BAR_BOX_COUNT=0
    if [[ $COLUMNS -ge 27 ]] ; then
      INITIAL_STR+="$(draw_box 26 "SYSTEM LOAD" 0)"
      STATUS_BAR_BOX_COUNT=1
      ARROW_POS=27
    fi
    if [[ $COLUMNS -ge 54 ]] ; then
      INITIAL_STR+="$(draw_box 26 "STATUS" 27)"
      STATUS_BAR_BOX_COUNT=2
      ARROW_POS=53
    fi
    if [[ $COLUMNS -ge 80 ]] ; then
      INITIAL_STR+="$(draw_box 26 "MODULES" 53)"
      STATUS_BAR_BOX_COUNT=3
      ARROW_POS=79
    fi
    if [[ $COLUMNS -ge 104 ]] ; then
      INITIAL_STR+="$(draw_box 26 "STATUS 2" 79)"
      STATUS_BAR_BOX_COUNT=4
    fi
 
    if [[ $STATUS_BAR_BOX_COUNT -lt 4 ]] ; then
      INITIAL_STR+="$(draw_arrows "$ARROW_POS")"
    fi
  fi
  if [[ -f "$STATUS_TMP_PATH" ]] ; then
    sed -i "1s/.*/$STATUS_BAR_BOX_COUNT/" "$STATUS_TMP_PATH" 2> /dev/null || true
  fi
  INITIAL_STR+="\e[H"
  # set cursor and boxes
  printf "%b" "$INITIAL_STR"
  box_updaters
}

预检查模块（所有以 P 开头的模块）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
#######################################################################################
# Pre-Check (P-modules)
#######################################################################################
if [[ "${PRE_CHECK}" -eq 1 ]] ; then
 
  print_ln "no_log"
  if [[ -d "${LOG_DIR}" ]]; then
    print_output "[!] Pre-checking phase started on ""$(date)""\\n""$(indent "${NC}""Firmware binary path: ""${FIRMWARE_PATH}")" "main"
  else
    print_output "[!] Pre-checking phase started on ""$(date)""\\n""$(indent "${NC}""Firmware binary path: ""${FIRMWARE_PATH}")" "no_log"
  fi
  write_notification "Pre-checking phase started"
 
  # 'main' functions of imported modules
  # in the pre-check phase we execute all modules with P[Number]_Name.sh
 
  run_modules "P" "${THREADED}" "0"
 
  # if we running threaded we ware going to wait for the slow guys here
  [[ ${THREADED} -eq 1 ]] && wait_for_pid "${WAIT_PIDS[@]}"
 
  print_ln "no_log"
 
  if [[ -d "${LOG_DIR}" ]]; then
    print_output "[!] Pre-checking phase ended on ""$(date)"" and took about ""$(date -d@"${SECONDS}" -u +%H:%M:%S)"" \\n" "main"
  else
    print_output "[!] Pre-checking phase ended on ""$(date)"" and took about ""$(date -d@"${SECONDS}" -u +%H:%M:%S)"" \\n" "no_log"
  fi
  write_notification "Pre-checking phase finished"
 
  # useful prints for debugging:
  # print_output "[!] Firmware value: ${FIRMWARE}"
  # print_output "[!] Firmware path: ${FIRMWARE}_PATH"
  # print_output "[!] Output dir: $OUTPUT_DIR"
  # print_output "[!] LINUX_PATH_COUNTER: $LINUX_PATH_COUNTER"
  # print_output "[!] LINUX_PATH_ARRAY: ${#ROOT_PATH[@]}"
fi

关键在run_modules "P" "${THREADED}" "0"这一行，我们看看这个函数的定义：
emba line 111：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# $1: module group letter [P, S, L, F]
# $2: 0=single thread 1=multithread
# $3: HTML=1 - generate html file
run_modules()
{
  MODULE_GROUP="${1:-}"
  printf -v THREADING_SET '%d\n' "${2}" 2>/dev/null
  THREADING_MOD_GROUP="${THREADING_SET}"
 
  local SELECT_PRE_MODULES_COUNT=0
 
  for SELECT_NUM in "${SELECT_MODULES[@]}" ; do
    if [[ "${SELECT_NUM}" =~ ^["${MODULE_GROUP,,}","${MODULE_GROUP^^}"]{1} ]]; then
...

函数比较长。接收三个参数的作用在声明前写的很清楚。大概功能：根据传入的参数和一些其他条件，动态地加载和运行符合要求的模块脚本，有时候可以进行多线程并发执行。另外，还会检查某个模块是否已经运行过，并在必要时跳过不需要重新运行的模块，提高运行效率。如果开启了 HTML 选项，则会生成报告文件。
固件检查模块（S模块）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#######################################################################################
# Firmware-Check (S modules)
#######################################################################################
WAIT_PIDS=()
if [[ ${FIRMWARE} -eq 1 ]] ; then
  print_output "\n=================================================================\n" "no_log"
 
  if [[ -d "${LOG_DIR}" ]]; then
    print_output "[!] Testing phase started on ""$(date)""\\n""$(indent "${NC}""Firmware path: ""${FIRMWARE_PATH}")" "main"
  else
    print_output "[!] Testing phase started on ""$(date)""\\n""$(indent "${NC}""Firmware path: ""${FIRMWARE_PATH}")" "no_log"
  fi
  write_notification "Testing phase finished"
  write_grep_log "$(date)" "TIMESTAMP"
 
  run_modules "S" "${THREADED}" "${HTML}"
 
  [[ ${THREADED} -eq 1 ]] && wait_for_pid "${WAIT_PIDS[@]}"
 
  print_ln "no_log"
 
  if [[ -d "${LOG_DIR}" ]]; then
    print_output "[!] Testing phase ended on ""$(date)"" and took about ""$(date -d@"${SECONDS}" -u +%H:%M:%S)"" \\n" "main"
  else
    print_output "[!] Testing phase ended on ""$(date)"" and took about ""$(date -d@"${SECONDS}" -u +%H:%M:%S)"" \\n" "no_log"
  fi
  write_notification "Testing phase ended"
 
  TESTING_DONE=1
fi

同理，会运行所有S模块，并等待子进程结束。必要时会使用多线程。
实时模拟模块（L模块）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#######################################################################################
# Live Emulation - Check (L-modules)
#######################################################################################
if [[ "${FULL_EMULATION}" -eq 1 ]] ; then
  print_output "\n=================================================================\n" "no_log"
  if [[ -d "${LOG_DIR}" ]]; then
    print_output "[!] System emulation phase started on ""$(date)""\\n""$(indent "${NC}""Firmware path: ""${FIRMWARE_PATH}")" "main"
  else
    print_output "[!] System emulation phase started on ""$(date)""\\n""$(indent "${NC}""Firmware path: ""${FIRMWARE_PATH}")" "no_log"
  fi
  write_notification "System emulation phase started"
 
  write_grep_log "$(date)" "TIMESTAMP"
  # these modules are not threaded!
  run_modules "L" "0" "${HTML}"
 
  print_ln "no_log"
  if [[ -d "${LOG_DIR}" ]]; then
    print_output "[!] System emulation phase ended on ""$(date)"" and took about ""$(date -d@"${SECONDS}" -u +%H:%M:%S)"" \\n" "main"
  else
    print_output "[!] System emulation ended on ""$(date)"" and took about ""$(date -d@"${SECONDS}" -u +%H:%M:%S)"" \\n" "no_log"
  fi
  write_notification "System emulation phase ended"
fi

会使用单线程运行。并运行所有L开头的模块。
报告模块（F模块）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
#######################################################################################
# Reporting (F-modules)
#######################################################################################
if [[ -d "${LOG_DIR}" ]]; then
  print_output "[!] Reporting phase started on ""$(date)""\\n" "main"
else
  print_output "[!] Reporting phase started on ""$(date)""\\n" "no_log"
fi
write_notification "Reporting phase started"
 
run_modules "F" "0" "${HTML}"
 
[[ ${DISABLE_STATUS_BAR} -eq 0 ]] && remove_status_bar
 
write_notification "Reporting phase ended"
 
if [[ "${TESTING_DONE}" -eq 1 ]]; then
  if [[ "${FINAL_FW_RM}" -eq 1 && -d "${LOG_DIR}"/firmware ]]; then
    print_output "[*] Removing temp firmware directory\\n" "no_log"
    rm -r "${LOG_DIR}"/firmware 2>/dev/null
  fi
  if [[ "${FINAL_FW_RM}" -eq 1 && -d "${LOG_DIR}"/p61_unblob_eval/unblob_extracted ]]; then
    print_output "[*] Removing unblob firmware directory\\n" "no_log"
    rm -r "${LOG_DIR}"/p61_unblob_eval/unblob_extracted 2>/dev/null
  fi
  print_ln "no_log"
  if [[ -d "${LOG_DIR}" ]]; then
    print_output "[!] Test ended on ""$(date)"" and took about ""$(date -d@"${SECONDS}" -u +%H:%M:%S)"" \\n" "main"
    write_notification "EMBA finished analysis"
    rm -r "${TMP_DIR}" 2>/dev/null || true
  else
    print_output "[!] Test ended on ""$(date)"" and took about ""$(date -d@"${SECONDS}" -u +%H:%M:%S)"" \\n" "no_log"
  fi
  write_grep_log "$(date)" "TIMESTAMP"
  write_grep_log "$(date -d@"${SECONDS}" -u +%H:%M:%S)" "DURATION"
else
  print_output "[!] No extracted firmware found" "no_log"
  print_output "$(indent "Try using binwalk or something else to extract the firmware")"
  exit 1
fi
 
[[ "${HTML}" -eq 1 ]] && update_index
 
if [[ -f "${HTML_PATH}"/index.html ]] && [[ "${IN_DOCKER}" -eq 0 ]]; then
  print_output "[*] Web report created HTML report in ${ORANGE}${LOG_DIR}/html-report${NC}\\n" "main"
  print_output "[*] Open the web-report with${ORANGE} firefox $(abs_path "${HTML_PATH}/index.html")${NC}\\n" "main"
fi
 
# we need to change the permissions of the LOG_DIR to the orig. user from the host
[[ "${IN_DOCKER}" -eq 1 ]] && restore_permissions
cleaner 0

单线程运行"F"模块的程序，用于处理测试结果并生成报告。如果指定了-B参数则删除状态栏。
在测试结束后，如果指定了-r参数则将产生的临时文件夹和文件删除（只有运行了S模块才会产生）。
总结：主运行文件将检查参数等，并调用各个模块：

  • 预检查模块（P 模块）
  • 固件检查模块（S模块）
  • 实时模拟模块（L模块）
  • 报告模块（F模块）

因此各个模块源代码的审计将是理解固件分析实现的重点。

主要模块梳理

预检查模块（P模块）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
P02_firmware_bin_file_check.sh    给出有关所提供的固件二进制文件的一些非常基本的信息。
P05_patools_init.sh    用patools工具提取zip、tar、tgz
P10_vmdk_extractor.sh    提取vmdk镜像
P11_dlink_SHRS_enc_extract.sh    提取D-link加密固件镜像
P12_avm_freetz_ng_extract.sh    使用Freetz-NG提取AVM固件映像
P13_uboot_mkimage.sh    显示Uboot映像的内部
P14_ext_mounter.sh    挂载和提取extX映像
P15_ubi_extractor.sh    提取ubi文件系统
P16_EnGenius_decryptor.sh    用EnGenius提取固件镜像
P17_gpg_decompress.sh    提取没有加密的gpg压缩的固件镜像
P18_qnap_decryptor.sh    从QNAP中提取加密固件映像
P19_bsd_ufs_mounter.sh    挂载和提取BSD UFS映像
P20_foscam_decryptor.sh    提取Foscam公司的加密固件映像
P21_buffalo_decryptor.sh    提取Buffalo公司的加密固件映像
P22_Zyxel_zip_decrypt.sh    提取被密码保护的Zyxel固件镜像
P23_qemu_qcow_mounter.sh    挂载和提取Qemu QCOW2镜像
P25_android_ota.sh    提取Android OTA更新文件
P35_UEFI_extractor.sh    提取UEFI镜像与BIOSUtilities
P59_binwalk_extractor.sh    分析固件与binwalk，检查熵和提取固件到日志目录
P60_firmware_bin_extractor.sh    和上面一样
P65_package_extractor.sh    识别和提取典型的软件包档案，如deb, apk, ipk
P70_unblob.sh    将带有unblob的固件提取到模块日志目录（仅用于评估）
P99_prepare_analyzer.sh    一些准备工作（检查固件、架构检查等）

固件检查模块（S模块）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
S02_UEFI_FwHunt.sh    使用FwHunt来识别可能的UEFI固件中的漏洞
S03_firmware_bin_base_analyzer.sh    对操作系统或架构进行识别
S05_firmware_details.sh    统计可执行文件
S06_distribution_identification.sh    识别发行版
S08_package_mgmt_extractor.sh    搜索已知位置的包管理信息
S09_firmware_base_version_check.sh    搜索二进制文件版本
S100_command_inj_check.sh    搜索web文件并搜索可以代码执行的东西
S106_deep_key_search.sh    搜素文件里的特殊字符串
S107_deep_password_search.sh    搜索文件里的特殊密码
S108_stacs_password_search.sh    使用stacs搜索密码
S109_jtr_local_pw_cracking.sh    破解文件哈希
S10_binaries_basic_check.sh    搜索二进制的风险
S110_yara_check.sh    用yara搜索
S115_usermode_emulator.sh    模拟执行固件获取版本信息（实验功能，需要-E单独开启）
S116_qemu_version_detection.sh    该模块从S115的结果中提取版本信息
S120_cwe_checker.sh    用Ghidra扫描二进制文件中的常见风险
S12_binary_protection.sh    搜索二进制文件的保护机制
S13_weak_func_check.sh    和S10类似
S14_weak_func_radare_check.sh    和S10类似，使用radare
S15_bootloader_check.sh    扫描设备树等
S17_apk_check.sh    识别apk包并用APKHunt扫漏洞
S20_shell_check.sh    扫描shell脚本的漏洞
S21_python_check.sh    扫描python脚本的漏洞
S22_php_check.sh    扫描php脚本的漏洞和php配置文件
S24_kernel_bin_identifier.sh    识别内核文件
S25_kernel_check.sh    扫漏洞，包括版本、描述、内核配置等
S26_kernel_vuln_verifier.sh    内核漏洞识别
S35_http_file_check.sh    搜索http相关文件
S36_lighttpd.sh    识别lighttpd配置文件等
S40_weak_perm_check.sh    检查系统用户相关配置、密码文件的风险
S45_pass_file_check.sh    检查密码相关文件
S50_authentication_check.sh    检查身份验证等
S55_history_file_check.sh    检查历史文件
S60_cert_file_check.sh    检查证书文件
S65_config_file_check.sh    扫描典型的配置文件
S70_hidden_file_check.sh    隐藏文件搜索
S75_network_check.sh    检查网络相关的配置文件
S80_cronjob_check.sh    检查所有定时配置文件
S85_ssh_check.sh    检查ssh相关配置文件
S90_mail_check.sh    搜素邮箱文件
S95_interesting_binaries_check.sh    搜索一些二进制文件
S99_grepit.sh    用grepit扫风险

实时模拟模块（L模块）

1
2
3
4
5
6
7
8
L10_system_emulation    文件夹
L10_system_emulation.sh    基于firmadyne和firmAE进行固件模拟（实验模块，需要-Q单独激活。）
L15_emulated_checks_nmap.sh    测试是否成功模拟
L20_snmp_checks.sh    同上
L25_web_checks.sh    测试模拟的web服务
L30_routersploit.sh    测试
L35_metasploit_check.sh    用metasploit测试
L99_cleanup.sh    停止并清理模拟环境

报告模块（F模块）

1
2
3
4
F10_license_summary.sh    收集许可证信息
F20_vul_aggregator.sh    收集漏洞
F21_cyclonedx_sbom.sh    生成json
F50_base_aggregator.sh    生成一个所有模块的概览

预检查模块（P模块）代码分析

P02_firmware_bin_file_check.sh

给出有关所提供的固件二进制文件的一些非常基本的信息。
主函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
P02_firmware_bin_file_check() {
  module_log_init "${FUNCNAME[0]}"
  module_title "Binary firmware file analyzer"
 
  set_p02_default_exports
 
  local FILE_BIN_OUT
  # we set this var global to 1 if we find something UEFI related
  export UEFI_DETECTED=0
 
  write_csv_log "Entity" "data" "Notes"
  write_csv_log "Firmware path" "$FIRMWARE_PATH" "NA"
  if [[ -f "$FIRMWARE_PATH" ]]; then
 
    # 计算固件文件哈希
    SHA512_CHECKSUM="$(sha512sum "$FIRMWARE_PATH" | awk '{print $1}')"
    write_csv_log "SHA512" "${SHA512_CHECKSUM:-}" "NA"
    SHA1_CHECKSUM="$(sha1sum "$FIRMWARE_PATH" | awk '{print $1}')"
    write_csv_log "SHA1" "${SHA1_CHECKSUM:-}" "NA"
    MD5_CHECKSUM="$(md5sum "$FIRMWARE_PATH" | awk '{print $1}')"
    write_csv_log "MD5" "${MD5_CHECKSUM:-}" "NA"
 
    # 使用 ent 计算文件的熵信息
    ENTROPY="$(ent "$FIRMWARE_PATH" | grep Entropy | sed -e 's/^Entropy\ \=\ //')"
    write_csv_log "Entropy" "${ENTROPY:-}" "NA"
 
    print_output "[*] Entropy testing with binwalk ... "
    # we have to change the working directory for binwalk, because everything except the log directory is read-only in
    # Docker container and binwalk fails to save the entropy picture there
    if [[ $IN_DOCKER -eq 1 ]] ; then
      cd "$LOG_DIR" || return
 
        # 使用 binwalk 提取固件，并输出 json 的结果
      print_output "$(binwalk -E -F -J "$FIRMWARE_PATH")"
      mv "$(basename "$FIRMWARE_PATH".png)" "$LOG_DIR"/firmware_entropy.png 2> /dev/null || true
      cd /emba || return
    else
      print_output "$(binwalk -E -F -J "$FIRMWARE_PATH")"
      mv "$(basename "$FIRMWARE_PATH".png)" "$LOG_DIR"/firmware_entropy.png 2> /dev/null || true
    fi
  fi
 
  local FILE_LS_OUT
  FILE_LS_OUT=$(ls -lh "$FIRMWARE_PATH")
 
  print_ln
  print_output "[*] Details of the firmware file:"
  print_ln
  print_output "$(indent "$FILE_LS_OUT")"
  print_ln
  if [[ -f "$FIRMWARE_PATH" ]]; then
    print_ln
 
      # 使用 file 命令提取基本信息
    print_output "$(indent "$(file "$FIRMWARE_PATH")")"
    print_ln
 
      # 使用 hexdump 输出文件前面的十六进制和ASCII信息
    hexdump -C "$FIRMWARE_PATH"| head | tee -a "$LOG_FILE" || true
    print_ln
    print_output "[*] SHA512 checksum: $ORANGE$SHA512_CHECKSUM$NC"
    print_ln
    print_output "$(indent "$ENTROPY")"
    print_ln
    if [[ -x "$EXT_DIR"/pixde ]]; then
 
      # 可视化二进制文件
      print_output "[*] Visualized firmware file (first 2000 bytes):\n"
      "$EXT_DIR"/pixde -r-0x2000 "$FIRMWARE_PATH" | tee -a "$LOG_DIR"/p02_pixd.txt
      print_ln
 
      python3 "$EXT_DIR"/pixd_png.py -i "$LOG_DIR"/p02_pixd.txt -o "$LOG_DIR"/pixd.png -p 10 > /dev/null
      write_link "$LOG_DIR"/pixd.png
    fi
 
      # 对二进制文件进行检测（将在下方进行分析）
    fw_bin_detector "$FIRMWARE_PATH"
 
      # 备份所有的变量
    backup_p02_vars
  fi
 
  module_end_log "${FUNCNAME[0]}" 1
}

fw_bin_detector函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
fw_bin_detector() {
 
    # 固件路径
  local CHECK_FILE="${1:-}"
  local FILE_BIN_OUT=""
  local DLINK_ENC_CHECK=""
  local QNAP_ENC_CHECK=""
  local AVM_CHECK=0
  local UEFI_CHECK=0
 
  set_p02_default_exports
 
  FILE_BIN_OUT=$(file "$CHECK_FILE")
 
    # 输出第一行用于检查是否为DLINK公司产品
  DLINK_ENC_CHECK=$(hexdump -C "$CHECK_FILE" | head -1 || true)
 
    # 从固件中所有的字符串中检查AVM正则出现的行数。用于判断固件是否为AVM公司产品。
  AVM_CHECK=$(strings "$CHECK_FILE" | grep -c "AVM GmbH .*. All rights reserved.\|(C) Copyright .* AVM" || true)
 
    # 搜索字符串并只显示包含指定字符串的结果，用于判断是否为QNAP加密文件系统
    QNAP_ENC_CHECK=$(binwalk -y "qnap encrypted" "$CHECK_FILE")
  # we are running binwalk on the file to analyze the output afterwards:
  binwalk "$CHECK_FILE" > "$TMP_DIR"/s02_binwalk_output.txt
  UEFI_CHECK=$(grep -c "UEFI" "$TMP_DIR"/s02_binwalk_output.txt || true)
  UEFI_CHECK=$(( "$UEFI_CHECK" + "$(grep -c "UEFI" "$CHECK_FILE" || true)" ))
 
  if [[ -f "$KERNEL_CONFIG" ]] && [[ "$KERNEL" -eq 1 ]]; then
    # we set the FIRMWARE_PATH to the kernel config path if we have only -k parameter
    if [[ "$(md5sum "$KERNEL_CONFIG" | awk '{print $1}')" == "$(md5sum "$FIRMWARE_PATH" | awk '{print $1}')" ]]; then
      print_output "[+] Identified Linux kernel configuration file"
      write_csv_log "kernel config" "yes" "NA"
      export SKIP_PRE_CHECKERS=1
      return
    fi
  fi
 
  if [[ "$UEFI_CHECK" -gt 0 ]]; then
    print_output "[+] Identified possible UEFI firmware - using fwhunt-scan vulnerability scanning module"
    export UEFI_DETECTED=1
    UEFI_AMI_CAPSULE=$(grep -c "AMI.*EFI.*capsule" "$TMP_DIR"/s02_binwalk_output.txt || true)
    if [[ "$UEFI_AMI_CAPSULE" -gt 0 ]]; then
      print_output "[+] Identified possible UEFI-AMI capsule firmware - using capsule extractors"
    fi
    write_csv_log "UEFI firmware detected" "yes" "NA"
  fi
  if [[ "$AVM_CHECK" -gt 0 ]] || [[ "$FW_VENDOR" == *"AVM"* ]]; then
    print_output "[+] Identified AVM firmware - using AVM extraction module"
    export AVM_DETECTED=1
    write_csv_log "AVM firmware detected" "yes" "NA"
  fi
  # if we have a zip, tgz, tar archive we are going to use the patools extractor
  if [[ "$FILE_BIN_OUT" == *"gzip compressed data"* || "$FILE_BIN_OUT" == *"Zip archive data"* || \
    "$FILE_BIN_OUT" == *"POSIX tar archive"* || "$FILE_BIN_OUT" == *"ISO 9660 CD-ROM filesystem data"* || \
    "$FILE_BIN_OUT" == *"7-zip archive data"* || "$FILE_BIN_OUT" == *"XZ compressed data"* || \
    "$FILE_BIN_OUT" == *"bzip2 compressed data"* ]]; then
    # as the AVM images are also zip files we need to bypass it here:
    if [[ "$AVM_DETECTED" -ne 1 ]]; then
      print_output "[+] Identified gzip/zip/tar/iso/xz/bzip2 archive file - using patools extraction module"
      export PATOOLS_INIT=1
      write_csv_log "basic compressed (patool)" "yes" "NA"
    fi
  fi
  if [[ "$FILE_BIN_OUT" == *"QEMU QCOW2 Image"* ]]; then
    print_output "[+] Identified Qemu QCOW image - using QCOW extraction module"
    export QCOW_DETECTED=1
    write_csv_log "Qemu QCOW firmware detected" "yes" "NA"
  fi
  if [[ "$FILE_BIN_OUT" == *"VMware4 disk image"* ]]; then
    print_output "[+] Identified VMWware VMDK archive file - using VMDK extraction module"
    export VMDK_DETECTED=1
    write_csv_log "VMDK" "yes" "NA"
  fi
  if [[ "$FILE_BIN_OUT" == *"UBI image"* ]]; then
    print_output "[+] Identified UBI filesystem image - using UBI extraction module"
    export UBI_IMAGE=1
    write_csv_log "UBI filesystem" "yes" "NA"
  fi
 
    # 识别DLINK固件
  if [[ "$DLINK_ENC_CHECK" == *"SHRS"* ]]; then
    print_output "[+] Identified D-Link SHRS encrpyted firmware - using D-Link extraction module"
    export DLINK_ENC_DETECTED=1
    write_csv_log "D-Link SHRS" "yes" "NA"
  fi
  if [[ "$DLINK_ENC_CHECK" =~ 00000000\ \ 00\ 00\ 00\ 00\ 00\ 00\ 0.\ ..\ \ 00\ 00\ 0.\ ..\ 31\ 32\ 33\ 00 ]]; then
    print_output "[+] Identified EnGenius encrpyted firmware - using EnGenius extraction module"
    export ENGENIUS_ENC_DETECTED=1
    write_csv_log "EnGenius encrypted" "yes" "NA"
  fi
  if [[ "$DLINK_ENC_CHECK" =~ 00000000\ \ 00\ 00\ 00\ 00\ 00\ 00\ 01\ 01\ \ 00\ 00\ 0.\ ..\ 33\ 2e\ 3[89]\ 2e ]]; then
    print_output "[+] Identified EnGenius encrpyted firmware - using EnGenius extraction module"
    export ENGENIUS_ENC_DETECTED=1
    write_csv_log "EnGenius encrypted" "yes" "NA"
  fi
  if [[ "$DLINK_ENC_CHECK" == *"encrpted_img"* ]]; then
    print_output "[+] Identified D-Link encrpted_img encrpyted firmware - using D-Link extraction module"
    export DLINK_ENC_DETECTED=2
    write_csv_log "D-Link encrpted_img encrypted" "yes" "NA"
  fi
 
    # 识别u-boot固件
  if [[ "$FILE_BIN_OUT" == *"u-boot legacy uImage"* ]]; then
    print_output "[+] Identified u-boot firmware - using u-boot module"
    export UBOOT_IMAGE=1
    write_csv_log "Uboot image" "yes" "NA"
  fi
  if [[ "$FILE_BIN_OUT" == *"Unix Fast File system [v2]"* ]]; then
    print_output "[+] Identified UFS filesytem - using UFS filesytem extraction module"
    export BSD_UFS=1
    write_csv_log "BSD UFS filesystem" "yes" "NA"
  fi
  if [[ "$FILE_BIN_OUT" == *"Linux rev 1.0 ext2 filesystem data"* ]]; then
    print_output "[+] Identified Linux ext2 filesytem - using EXT filesytem extraction module"
    export EXT_IMAGE=1
    write_csv_log "EXT2 filesystem" "yes" "NA"
  fi
  if [[ "$FILE_BIN_OUT" == *"Linux rev 1.0 ext3 filesystem data"* ]]; then
    print_output "[+] Identified Linux ext3 filesytem - using EXT filesytem extraction module"
    export EXT_IMAGE=1
    write_csv_log "EXT3 filesystem" "yes" "NA"
  fi
  if [[ "$FILE_BIN_OUT" == *"Linux rev 1.0 ext4 filesystem data"* ]]; then
    print_output "[+] Identified Linux ext4 filesytem - using EXT filesytem extraction module"
    export EXT_IMAGE=1
    write_csv_log "EXT4 filesystem" "yes" "NA"
  fi
  if [[ "$QNAP_ENC_CHECK" == *"QNAP encrypted firmware footer , model"* ]]; then
    print_output "[+] Identified QNAP encrpyted firmware - using QNAP extraction module"
    export QNAP_ENC_DETECTED=1
    write_csv_log "QNAP encrypted filesystem" "yes" "NA"
  fi
  # probably we need to take a deeper look to identify the gpg compressed firmware files better.
  # Currently this detection mechanism works quite good on the known firmware images
  if [[ "$DLINK_ENC_CHECK" =~ 00000000\ \ a3\ 01\  ]]; then
    GPG_CHECK="$(gpg --list-packets "$FIRMWARE_PATH" | grep "compressed packet:")"
    if [[ "$GPG_CHECK" == *"compressed packet: algo="* ]]; then
      print_output "[+] Identified GPG compressed firmware - using GPG extraction module"
      export GPG_COMPRESS=1
      write_csv_log "GPG compressed firmware" "yes" "NA"
    fi
  fi
  if [[ "$DLINK_ENC_CHECK" == *"CrAU"* ]]; then
    print_output "[+] Identified Android OTA payload.bin update file - using Android extraction module"
    export ANDROID_OTA=1
    write_csv_log "Android OTA update" "yes" "NA"
  fi
  if [[ "$FILE_BIN_OUT" == *"openssl enc'd data with salted password"* ]]; then
    print_output "[+] Identified OpenSSL encrypted file - trying OpenSSL module for Foscam firmware"
    export OPENSSL_ENC_DETECTED=1
    write_csv_log "OpenSSL encrypted" "yes" "NA"
  fi
  # This check is currently only tested on one firmware - further tests needed:
  if [[ "$DLINK_ENC_CHECK" =~ 00000000\ \ 62\ 67\ 6e\ 00\ 00\ 00\ 00\ 00\ \ 00\ 00\ 00\ b9\ 01\  ]]; then
    print_output "[+] Identified Buffalo encrpyted firmware - using Buffalo extraction module"
    export BUFFALO_ENC_DETECTED=1
    write_csv_log "Buffalo encrypted" "yes" "NA"
  fi
  if [[ "$(basename "$CHECK_FILE")" =~ .*\.ri ]] && [[ "$FILE_BIN_OUT" == *"data"* ]]; then
    # ri files are usually used by zyxel
    if [[ $(find "$LOG_DIR"/firmware -name "$(basename -s .ri "$CHECK_FILE")".bin | wc -l) -gt 0 ]]; then
      # if we find a bin file with the same name then it is a Zyxel firmware image
      print_output "[+] Identified ZyXel encrpyted ZIP firmware - using ZyXel extraction module"
      export ZYXEL_ZIP=1
      write_csv_log "ZyXel encrypted ZIP" "yes" ""
    fi
  fi
  print_ln
}

P05_patools_init.sh

用patools工具提取zip、tar、tgz

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
export PRE_THREAD_ENA=0
 
P05_patools_init() {
  local NEG_LOG=0
 
  if [[ "$PATOOLS_INIT" -eq 1 ]]; then
    module_log_init "${FUNCNAME[0]}"
    module_title "Initial extractor of different archive types via patools"
    pre_module_reporter "${FUNCNAME[0]}"
 
    EXTRACTION_DIR="$LOG_DIR"/firmware/patool_extraction/
 
    patools_extractor "$FIRMWARE_PATH" "$EXTRACTION_DIR"
 
    if [[ "$FILES_PATOOLS" -gt 0 ]]; then
      MD5_DONE_DEEP+=( "$(md5sum "$FIRMWARE_PATH" | awk '{print $1}')" )
      export FIRMWARE_PATH="$LOG_DIR"/firmware/
      backup_var "FIRMWARE_PATH" "$FIRMWARE_PATH"
    fi
 
    NEG_LOG=1
    module_end_log "${FUNCNAME[0]}" "$NEG_LOG"
  fi
}
 
patools_extractor() {
  sub_module_title "Patool filesystem extractor"
 
  local FIRMWARE_PATH_="${1:-}"
  local EXTRACTION_DIR_="${2:-}"
  FILES_PATOOLS=0
  local DIRS_PATOOLS=0
  local FIRMWARE_NAME_=""
 
  if ! [[ -f "$FIRMWARE_PATH_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
  FIRMWARE_NAME_="$(basename "$FIRMWARE_PATH_")"
 
  [[ "$STRICT_MODE" -eq 1 ]] && set +e
 
    # 使用patool测试
  patool -v test "$FIRMWARE_PATH_" | tee -a "$LOG_PATH_MODULE"/paextract_test_"$FIRMWARE_NAME_".log
 
  [[ "$STRICT_MODE" -eq 1 ]] && set -e
 
  cat "$LOG_PATH_MODULE"/paextract_test_"$FIRMWARE_NAME_".log >> "$LOG_FILE"
 
  if ! [[ -d "$EXTRACTION_DIR_" ]]; then
    mkdir "$EXTRACTION_DIR_"
  fi
 
  if grep -q "patool: ... tested ok." "$LOG_PATH_MODULE"/paextract_test_"$FIRMWARE_NAME_".log ; then
 
    print_ln
    print_output "[*] Valid compressed file detected - extraction process via patool started"
 
      # 提取
    patool -v extract "$FIRMWARE_PATH_" --outdir "$EXTRACTION_DIR_" | tee -a "$LOG_PATH_MODULE"/paextract_extract_"$FIRMWARE_NAME_".log
    cat "$LOG_PATH_MODULE"/paextract_extract_"$FIRMWARE_NAME_".log >> "$LOG_FILE"
 
  else
    # Fallback if unzip does not work:
    print_ln
    print_output "[*] No valid compressed file detected - extraction process via binwalk started"
 
      # P60里的函数，下方给出分析
    binwalk_deep_extract_helper 0 "$FIRMWARE_PATH_" "$EXTRACTION_DIR_"
  fi
 
  print_ln
  print_output "[*] Using the following firmware directory ($ORANGE$EXTRACTION_DIR_$NC) as base directory:"
  find "$EXTRACTION_DIR_" -xdev -maxdepth 1 -ls | tee -a "$LOG_FILE"
  print_ln
 
  FILES_PATOOLS=$(find "$EXTRACTION_DIR_" -type f | wc -l)
  DIRS_PATOOLS=$(find "$EXTRACTION_DIR_" -type d | wc -l)
  print_output "[*] Extracted $ORANGE$FILES_PATOOLS$NC files and $ORANGE$DIRS_PATOOLS$NC directories from the firmware image."
  write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
  write_csv_log "Patool extractor" "$FIRMWARE_PATH_" "$EXTRACTION_DIR_" "$FILES_PATOOLS" "$DIRS_PATOOLS" "NA"
  print_ln
}

binwalk_deep_extract_helper函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
binwalk_deep_extract_helper() {
  # Matryoshka mode is first parameter: 1 - enable, 0 - disable
  local MATRYOSHKA_="${1:-0}"
  local FILE_TO_EXTRACT_="${2:-}"
  local DEST_FILE_="${3:-}"
 
  if ! [[ -f "$FILE_TO_EXTRACT_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
    # 前面通过检查binwalk的版本，使用不同方法提取
  if [[ "$BINWALK_VER_CHECK" == 1 ]]; then
    if [[ "$MATRYOSHKA_" -eq 1 ]]; then
 
        # 使用binwalk提取
      binwalk --run-as=root --preserve-symlinks --dd='.*' -e -M -C "$DEST_FILE_" "$FILE_TO_EXTRACT_" | tee -a "$LOG_FILE" || true
    else
      # no more Matryoshka mode ... we are doing it manually and check the files every round via MD5
      binwalk --run-as=root --preserve-symlinks --dd='.*' -e -C "$DEST_FILE_" "$FILE_TO_EXTRACT_" | tee -a "$LOG_FILE" || true
    fi
  else
    if [[ "$MATRYOSHKA_" -eq 1 ]]; then
      binwalk --dd='.*' -e -M -C "$DEST_FILE_" "$FILE_TO_EXTRACT_" | tee -a "$LOG_FILE" || true
    else
      binwalk --dd='.*' -e -C "$DEST_FILE_" "$FILE_TO_EXTRACT_" | tee -a "$LOG_FILE" || true
    fi
  fi
}

P10_vmdk_extractor.sh

提取vmdk镜像

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
P10_vmdk_extractor() {
  local NEG_LOG=0
 
  if [[ "${VMDK_DETECTED-0}" -eq 1 ]]; then
    module_log_init "${FUNCNAME[0]}"
    module_title "VMDK (Virtual Machine Disk) extractor"
    EXTRACTION_DIR="$LOG_DIR"/firmware/vmdk_extractor/
 
      # 提取vmdk
    vmdk_extractor "$FIRMWARE_PATH" "$EXTRACTION_DIR"
 
    if [[ "$VMDK_FILES" -gt 0 ]]; then
      MD5_DONE_DEEP+=( "$(md5sum "$FIRMWARE_PATH" | awk '{print $1}')" )
      export FIRMWARE_PATH="$LOG_DIR"/firmware/
      backup_var "FIRMWARE_PATH" "$FIRMWARE_PATH"
    fi
    NEG_LOG=1
    module_end_log "${FUNCNAME[0]}" "$NEG_LOG"
  fi
}

vmdk_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
vmdk_extractor() {
  local VMDK_PATH_="${1:-}"
  local EXTRACTION_DIR_="${2:-}"
  local MOUNT_DEV=""
  local DEV_NAME=""
  local TMP_VMDK_MNT="$TMP_DIR/vmdk_mount_$RANDOM"
  local VMDK_DIRS=0
  local RET=0
  VMDK_FILES=0
 
  if ! [[ -f "$VMDK_PATH_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
    # 设置子模块title
  sub_module_title "VMDK (Virtual Machine Disk) extractor"
 
  print_output "[*] Enumeration of devices in VMDK images $ORANGE$VMDK_PATH_$NC"
  disable_strict_mode "$STRICT_MODE" 0
 
  # 使用virt工具输出文件系统信息
  virt-filesystems -a "$VMDK_PATH_" > "$TMP_DIR"/vmdk.log
  RET="$?"
 
  if [[ "$RET" -ne 0 ]]; then
 
      # 用7z解压vmdk（7z挺强大）
    # backup with 7z
    7z x -o"$EXTRACTION_DIR_" "$VMDK_PATH_"
 
      # 获取7z结果返回的状态码，并进行错误处理
    RET="$?"
    if [[ "$RET" -ne 0 ]]; then
      print_output "[-] WARNING: VMDK filesystem not enumerated"
      enable_strict_mode "$STRICT_MODE" 0
      return
    fi
  else
    mapfile -t VMDK_VIRT_FS < "$TMP_DIR"/vmdk.log
    for MOUNT_DEV in "${VMDK_VIRT_FS[@]}"; do
      print_output "[*] Found device $ORANGE$MOUNT_DEV$NC"
    done
  fi
  enable_strict_mode "$STRICT_MODE" 0
 
  mkdir -p "$TMP_VMDK_MNT" || true
 
  for MOUNT_DEV in "${VMDK_VIRT_FS[@]}"; do
    DEV_NAME=$(basename "$MOUNT_DEV")
 
      # 尝试挂载vmdk
    print_output "[*] Trying to mount $ORANGE$MOUNT_DEV$NC to $ORANGE$TMP_VMDK_MNT$NC directory"
    # if troubles ahead with vmdk mount, remove the error redirection
    guestmount -a "$VMDK_PATH_" -m "$MOUNT_DEV" --ro "$TMP_VMDK_MNT" 2>/dev/null || true
    if mount | grep -q vmdk_mount; then
      print_output "[*] Copying $ORANGE$MOUNT_DEV$NC to firmware directory $ORANGE$EXTRACTION_DIR_/$DEV_NAME$NC"
      mkdir -p "$EXTRACTION_DIR_"/"$DEV_NAME"/ || true
      cp -pri "$TMP_VMDK_MNT"/* "$EXTRACTION_DIR_"/"$DEV_NAME"/ || true
      umount "$TMP_VMDK_MNT"
    fi
  done
 
    # 统计提取出的文件和文件夹数量
  if [[ -d "$EXTRACTION_DIR_" ]]; then
    VMDK_FILES=$(find "$EXTRACTION_DIR_" -type f | wc -l)
    VMDK_DIRS=$(find "$EXTRACTION_DIR_" -type d | wc -l)
  fi
 
  # 输出结果
  if [[ "$VMDK_FILES" -gt 0 ]]; then
    print_ln
    print_output "[*] Extracted $ORANGE$VMDK_FILES$NC files and $ORANGE$VMDK_DIRS$NC directories from the firmware image."
    write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
    write_csv_log "VMDK extractor" "$VMDK_PATH_" "$EXTRACTION_DIR_" "$VMDK_FILES" "$VMDK_DIRS" "NA"
    # currently unblob has issues with VMDKs. We need to disable it for this extraction process
    safe_echo 0 > "$TMP_DIR"/unblob_disable.cfg
  fi
 
    # 删除临时用于挂载的文件夹
  rm -r "$TMP_VMDK_MNT" || true
}

P11_dlink_SHRS_enc_extract.sh

提取D-link加密固件镜像

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
P11_dlink_SHRS_enc_extract() {
  local NEG_LOG=0
 
  if [[ "$DLINK_ENC_DETECTED" -ne 0 ]] && [[ "$MODULE_DISABLED" -ne 1 ]]; then
    module_log_init "${FUNCNAME[0]}"
    module_title "DLink encrypted firmware extractor"
    pre_module_reporter "${FUNCNAME[0]}"
    EXTRACTION_FILE="$LOG_DIR"/firmware/firmware_dlink_dec.bin
 
      # 有不同的加密方式，使用对应的方式解密
    if [[ "$DLINK_ENC_DETECTED" -eq 1 ]]; then
      dlink_SHRS_enc_extractor "$FIRMWARE_PATH" "$EXTRACTION_FILE"n=
    elif [[ "$DLINK_ENC_DETECTED" -eq 2 ]]; then
      dlink_enc_img_extractor "$FIRMWARE_PATH" "$EXTRACTION_FILE"
    fi
 
    NEG_LOG=1
    module_end_log "${FUNCNAME[0]}" "$NEG_LOG"
  fi
}

dlink_SHRS_enc_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
dlink_SHRS_enc_extractor() {
  local DLINK_ENC_PATH_="${1:-}"
  local EXTRACTION_FILE_="${2:-}"
  if ! [[ -f "$DLINK_ENC_PATH_" ]]; then
    print_output "[-] No file for decryption provided"
    return
  fi
 
  local MODULE_DISABLED=1
 
  if [[ "$MODULE_DISABLED" -eq 1 ]]; then
    print_output "[*] Module ${FUNCNAME[0]} is deprecated and will be removed in the future"
    return
  fi
 
  sub_module_title "DLink encrypted firmware extractor"
 
  hexdump -C "$DLINK_ENC_PATH_" | head | tee -a "$LOG_FILE" || true
 
  print_ln
 
    # 使用DLINK密钥对固件进行AES解密
    # 参考：https://www.bleepingcomputer.com/news/security/d-link-blunder-firmware-encryption-key-exposed-in-unencrypted-image/
  dd if="$DLINK_ENC_PATH_" skip=1756 iflag=skip_bytes|openssl aes-128-cbc -d -p -nopad -nosalt -K "c05fbf1936c99429ce2a0781f08d6ad8" -iv "67c6697351ff4aec29cdbaabf2fbe346" --nosalt -in /dev/stdin -out "$EXTRACTION_FILE_" 2>&1 || true | tee -a "$LOG_FILE"
 
  print_ln
  if [[ -f "$EXTRACTION_FILE_" ]]; then
    print_output "[+] Decrypted D-Link firmware file to $ORANGE$EXTRACTION_FILE_$NC"
    print_ln
    print_output "[*] Firmware file details: $ORANGE$(file "$EXTRACTION_FILE_")$NC"
    write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
    write_csv_log "DLink SHRS decryptor" "$DLINK_ENC_PATH_" "$EXTRACTION_FILE_" "1" "NA" "NA"
    export FIRMWARE_PATH="$EXTRACTION_FILE_"
    backup_var "FIRMWARE_PATH" "$FIRMWARE_PATH"
    if [[ -z "${FW_VENDOR:-}" ]]; then
      FW_VENDOR="D-Link"
      backup_var "FW_VENDOR" "$FW_VENDOR"
    fi
  else
    print_output "[-] Decryption of D-Link firmware file failed"
  fi
}

dlink_enc_img_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
dlink_enc_img_extractor(){
  local TMP_DIR="$LOG_DIR""/tmp"
  local DLINK_ENC_PATH_="${1:-}"
  local EXTRACTION_FILE_="${2:-}"
  local TMP_IMAGE_FILE="$TMP_DIR/image.bin"
  if ! [[ -f "$DLINK_ENC_PATH_" ]]; then
    print_output "[-] No file for decryption provided"
    return
  fi
  local IMAGE_SIZE=0
  local OFFSET=0
  local ITERATION=0
 
  local MODULE_DISABLED=1
 
  if [[ "$MODULE_DISABLED" -eq 1 ]]; then
    print_output "[*] Module ${FUNCNAME[0]} is deprecated and will be removed in the future"
    return
  fi
 
  sub_module_title "DLink encrpted_image extractor"
 
  hexdump -C "$DLINK_ENC_PATH_" | head | tee -a "$LOG_FILE" || true
 
    # 从偏移量为 16 的位置开始复制到另外一个文件 $TMP_IMAGE_FILE 中
  dd if="$DLINK_ENC_PATH_" skip=16 iflag=skip_bytes of="$TMP_IMAGE_FILE" 2>&1 | tee -a "$LOG_FILE"
 
  IMAGE_SIZE=$(stat -c%s "$TMP_IMAGE_FILE")
  (( ROOF=IMAGE_SIZE/131072 ))
 
    # 每 131072（128KB） 为一块进行处理
  for ((ITERATION=0; ITERATION<ROOF; ITERATION++)); do
    if [[ "$ITERATION" -eq 0 ]]; then
      OFFSET=0
    else
      (( OFFSET=131072*ITERATION ))
    fi
 
      # aes解密
    dd if="$TMP_IMAGE_FILE" skip="$OFFSET" iflag=skip_bytes count=256| openssl aes-256-cbc -d -in /dev/stdin  -out /dev/stdout \
    -K "6865392d342b4d212964363d6d7e7765312c7132613364316e26322a5a5e2538" -iv "4a253169516c38243d6c6d2d3b384145" --nopad \
    --nosalt | dd if=/dev/stdin of="$EXTRACTION_FILE_" oflag=append conv=notrunc 2>&1 | tee -a "$LOG_FILE"
  done
  # Now it should be a .ubi file thats somewhat readable and extractable via ubireader
  print_ln
  if [[ -f "$EXTRACTION_FILE_" ]]; then
    UBI_OUT=$(file "$EXTRACTION_FILE_")
    if [[ "$UBI_OUT" == *"UBI image, version"* ]]; then
      print_output "[+] Decrypted D-Link firmware file to $ORANGE$EXTRACTION_FILE_$NC"
      print_ln
      print_output "[*] Firmware file details: $ORANGE$(file "$EXTRACTION_FILE_")$NC"
      write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
      write_csv_log "DLink enc_img decryptor" "$DLINK_ENC_PATH_" "$EXTRACTION_FILE_" "1" "NA" "NA"
      export FIRMWARE_PATH="$EXTRACTION_FILE_"
      backup_var "FIRMWARE_PATH" "$FIRMWARE_PATH"
      if [[ -z "${FW_VENDOR:-}" ]]; then
        FW_VENDOR="D-Link"
        backup_var "FW_VENDOR" "$FW_VENDOR"
      fi
      EXTRACTION_DIR="$LOG_DIR/firmware/dlink_ubi_extracted"
      mkdir -p "$EXTRACTION_DIR" || true
 
        # ubi提取
      ubi_extractor "$FIRMWARE_PATH" "$EXTRACTION_DIR"
    else
      print_output "[-] Further extraction of D-Link firmware file via deep extraction"
    fi
  else
    print_output "[-] Decryption of D-Link firmware file failed"
  fi
}

P12_avm_freetz_ng_extract.sh

使用Freetz-NG提取AVM固件映像
avm_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
avm_extractor() {
  local AVM_FW_PATH_="${1:-}"
  local EXTRACTION_DIR_="${2:-}"
  if ! [[ -f "$AVM_FW_PATH_" ]]; then
    return
  fi
  local FRITZ_DIRS=0
  local FIT_IMAGES=()
  local FIT_IMAGE=""
  local RAM_DISKS=()
  local RAM_DISK=""
  local RAM_DISK_NAME=""
  export FRITZ_FILE=0
  export FRITZ_VERSION=""
 
  local MODULE_DISABLED=1
 
  if [[ "$MODULE_DISABLED" -eq 1 ]]; then
    print_output "[*] Module ${FUNCNAME[0]} is deprecated and will be removed in the future"
    return
  fi
 
  sub_module_title "AVM freetz-ng firmware extractor"
 
  # read only filesystem bypass:
    # 将 freetz-ng 工具链的 .config 文件复制到临时目录中
  cp "$EXT_DIR"/freetz-ng/.config "$TMP_DIR"/.config
    # 使用 freetz-ng/fwmod 工具从 AVM 固件中提取文件
  "$EXT_DIR"/freetz-ng/fwmod -u -i "$TMP_DIR"/.config -d "$EXTRACTION_DIR_" "$AVM_FW_PATH_" | tee -a "$LOG_FILE" || true
 
  if [[ -d "$EXTRACTION_DIR_" ]]; then
 
      # FRITZ 是来自德国 AVM 公司生产的一系列家用网络设备，包括路由器、网关、电话和其他相关产品。
      # 统计提取出的文件和文件夹数量
    FRITZ_FILES=$(find "$EXTRACTION_DIR_" -type f | wc -l)
    FRITZ_DIRS=$(find "$EXTRACTION_DIR_" -type d | wc -l)
      # 提取版本信息
    FRITZ_VERSION=$(grep "detected firmware version:" "$LOG_FILE" | cut -d ":" -f2- || true)
    if [[ -z "$FRITZ_VERSION" ]]; then
      FRITZ_VERSION="NA"
    else
      print_output "[+] Detected Fritz version: $ORANGE$FRITZ_VERSION$NC"
    fi
 
    # fitimages are handled here with fitimg - binwalk and unblob are also able to handle these images
    # but it is currently more beautiful doing the AVM extraction in one place here
    mapfile -t FIT_IMAGES < <(find "$EXTRACTION_DIR_" -type f -name "fit-image")
      # 在提取的文件中，该函数会检测是否存在名为 "fit-image" 的文件
        # 如果存在则调用 fitimg 工具对其解压缩
        # 解压缩结果输出到 EXTRACTION_DIR/fit-image-extraction 目录中
        # 并从中提取 RAM_DISK
    if [[ "${#FIT_IMAGES[@]}" -gt 0 ]]; then
      if [[ -f "$EXT_DIR"/fitimg-0.8/fitimg ]]; then
        for FIT_IMAGE in "${FIT_IMAGES[@]}"; do
          print_output "[*] Detected fit-image: $ORANGE$FIT_IMAGE$NC"
          print_output "[*] Extracting fit-image with fitimg to $ORANGE$EXTRACTION_DIR/fit-image-extraction$NC"
          mkdir -p "$EXTRACTION_DIR/fit-image-extraction"
          "$EXT_DIR"/fitimg-0.8/fitimg -x "$FIT_IMAGE" -d "$EXTRACTION_DIR"/fit-image-extraction || true
          mapfile -t RAM_DISKS < <(find "$EXTRACTION_DIR_"/fit-image-extraction -type f -name "*ramdisk")
          print_ln
        done
      else
        print_output "[-] Fitimg installation not available - check your installation"
      fi
    fi
    if [[ "${#RAM_DISKS[@]}" -gt 0 ]]; then
      for RAM_DISK in "${RAM_DISKS[@]}"; do
        print_output "[*] Detected AVM ramdisk: $ORANGE$RAM_DISK$NC"
        RAM_DISK_NAME="$(basename "$RAM_DISK")"
 
          # 对RAM_DISK进行深度提取
        binwalk_deep_extract_helper 1 "$RAM_DISK" "$EXTRACTION_DIR_"/fit-image-extraction/"$RAM_DISK_NAME"_binwalk
        print_ln
      done
    fi
 
    if [[ "$FRITZ_FILES" -gt 0 ]]; then
      print_ln
      print_output "[*] Extracted $ORANGE$FRITZ_FILES$NC files and $ORANGE$FRITZ_DIRS$NC directories from the firmware image."
      write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
      write_csv_log "AVM extractor" "$AVM_FW_PATH_" "$EXTRACTION_DIR_" "$FRITZ_FILES" "$FRITZ_DIRS" "$FRITZ_VERSION"
      export DEEP_EXTRACTOR=1
      MD5_DONE_DEEP+=( "$(md5sum "$AVM_FW_PATH_" | awk '{print $1}')" )
 
      if [[ -z "${FW_VENDOR:-}" ]]; then
        FW_VENDOR="AVM"
        backup_var "FW_VENDOR" "$FW_VENDOR"
      fi
      if [[ -z "${FW_VERSION:-}" && "$FRITZ_VERSION" != "NA" ]]; then
        FW_VERSION="$FRITZ_VERSION"
      fi
    fi
  fi
}

P13_uboot_mkimage.sh

显示Uboot映像的内部
P13_uboot_mkimage函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
P13_uboot_mkimage() {
  local NEG_LOG=0
  if [[ "$UBOOT_IMAGE" -eq 1 ]]; then
    module_log_init "${FUNCNAME[0]}"
    local IMAGE_NAME=""
    local IMAGE_TYPE=""
    module_title "Uboot image details"
    pre_module_reporter "${FUNCNAME[0]}"
 
      # 使用mkimage工具读取固件
    mkimage -l "$FIRMWARE_PATH" | tee -a "$LOG_FILE"
    IMAGE_NAME=$(grep "Image Name" "$LOG_FILE" 2>/dev/null | awk '{print $3,$4,$5,$6,$7,$8,$9,$10}' || true)
    IMAGE_TYPE=$(grep "Image Type" "$LOG_FILE" 2>/dev/null | awk '{print $3,$4,$5,$6,$7,$8,$9,$10}' || true)
    write_csv_log "Identifier" "Value"
    write_csv_log "ImageName" "$IMAGE_NAME"
    write_csv_log "ImageType" "$IMAGE_TYPE"
    NEG_LOG=1
    module_end_log "${FUNCNAME[0]}" "$NEG_LOG"
  fi
}

P14_ext_mounter.sh

挂载和提取extX映像
ext_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
ext_extractor() {
  local EXT_PATH_="${1:-}"
  local EXTRACTION_DIR_="${2:-}"
  local TMP_EXT_MOUNT="$TMP_DIR""/ext_mount_$RANDOM"
  local DIRS_EXT_MOUNT=0
  FILES_EXT_MOUNT=0
 
  if ! [[ -f "$EXT_PATH_" ]]; then
    print_output "[-] No file for decryption provided"
    return
  fi
 
  sub_module_title "EXT filesystem extractor"
 
  mkdir -p "$TMP_EXT_MOUNT" || true
  print_output "[*] Trying to mount $ORANGE$EXT_PATH_$NC to $ORANGE$TMP_EXT_MOUNT$NC directory"
  mount -o ro "$EXT_PATH_" "$TMP_EXT_MOUNT"
  if mount | grep -q ext_mount; then
    print_output "[*] Copying $ORANGE$TMP_EXT_MOUNT$NC to firmware tmp directory ($EXTRACTION_DIR_)"
    mkdir -p "$EXTRACTION_DIR_"
    cp -pri "$TMP_EXT_MOUNT"/* "$EXTRACTION_DIR_"
    print_ln
    print_output "[*] Using the following firmware directory ($ORANGE$EXTRACTION_DIR_$NC) as base directory:"
    find "$EXTRACTION_DIR_" -xdev -maxdepth 1 -ls | tee -a "$LOG_FILE"
    print_ln
    print_output "[*] Unmounting $ORANGE$TMP_EXT_MOUNT$NC directory"
 
    FILES_EXT_MOUNT=$(find "$EXTRACTION_DIR_" -type f | wc -l)
    DIRS_EXT_MOUNT=$(find "$EXTRACTION_DIR_" -type d | wc -l)
    print_output "[*] Extracted $ORANGE$FILES_EXT_MOUNT$NC files and $ORANGE$DIRS_EXT_MOUNT$NC directories from the firmware image."
    write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
    write_csv_log "EXT filesystem extractor" "$EXT_PATH_" "$EXTRACTION_DIR_" "$FILES_EXT_MOUNT" "$DIRS_EXT_MOUNT" "NA"
    umount "$TMP_EXT_MOUNT" || true
  fi
  rm -r "$TMP_EXT_MOUNT"
}

P15_ubi_extractor.sh

提取ubi文件系统
ubi_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
ubi_extractor() {
  local UBI_PATH_="${1:-}"
  local EXTRACTION_DIR_="${2:-}"
  local UBI_FILE=""
  local UBI_INFO=""
  local UBI_1st_ROUND=""
  local UBI_DATA=""
  local DIRS_UBI_EXT=0
  FILES_UBI_EXT=0
  if ! [[ -f "$UBI_PATH_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
  sub_module_title "UBI filesystem extractor"
 
  print_output "[*] Extracts UBI firmware image $ORANGE$UBI_PATH_$NC with ${ORANGE}ubireader_extract_images$NC."
  print_output "[*] File details: $ORANGE$(file "$UBI_PATH_" | cut -d ':' -f2-)$NC"
 
  # 提取ubi镜像
    # 工具仓库：https://github.com/jrspruitt/ubi_reader
  ubireader_extract_images -i -v -w -o "$EXTRACTION_DIR_"/ubi_images "$UBI_PATH_" | tee -a "$LOG_FILE" || true
  FILES_UBI_EXT=$(find "$EXTRACTION_DIR_"/ubi_images -type f | wc -l)
  DIRS_UBI_EXT=$(find "$EXTRACTION_DIR_"/ubi_images -type d | wc -l)
  print_output "[*] Extracted $ORANGE$FILES_UBI_EXT$NC files and $ORANGE$DIRS_UBI_EXT$NC directories from the firmware image via UBI extraction round 1."
 
  print_output "[*] Extracts UBI firmware image $ORANGE$UBI_PATH_$NC with ${ORANGE}ubireader_extract_files$NC."
 
  ubireader_extract_files -i -v -w -o "$EXTRACTION_DIR_"/ubi_files "$UBI_PATH_" | tee -a "$LOG_FILE" || true
  FILES_UBI_EXT=$(find "$EXTRACTION_DIR_"/ubi_files -type f | wc -l)
  DIRS_UBI_EXT=$(find "$EXTRACTION_DIR_"/ubi_files -type d | wc -l)
  print_output "[*] Extracted $ORANGE$FILES_UBI_EXT$NC files and $ORANGE$DIRS_UBI_EXT$NC directories from the firmware image via UBI extraction round 2."
 
    # 深度提取，继续查找UBI文件镜像，然后提取
  if [[ -d "$EXTRACTION_DIR_" ]]; then
    mapfile -t UBI_1st_ROUND < <(find "$EXTRACTION_DIR_" -type f -exec file {} \; | grep "UBI image" || true)
 
    for UBI_DATA in "${UBI_1st_ROUND[@]}"; do
      UBI_FILE=$(safe_echo "$UBI_DATA" | cut -d: -f1)
      UBI_INFO=$(safe_echo "$UBI_DATA" | cut -d: -f2)
      if [[ "$UBI_INFO" == *"UBIfs image"* ]]; then
        sub_module_title "UBIfs deep extraction"
        print_output "[*] Extracts UBIfs firmware image $ORANGE$UBI_PATH_$NC with ${ORANGE}ubireader_extract_files$NC."
        print_output "[*] File details: $ORANGE$(file "$UBI_FILE" | cut -d ':' -f2-)$NC"
        ubireader_extract_files -l -i -w -v -o "$EXTRACTION_DIR_"/UBIfs_extracted "$UBI_FILE" | tee -a "$LOG_FILE" || true
        FILES_UBI_EXT=$(find "$EXTRACTION_DIR_"/UBIfs_extracted -type f | wc -l)
        DIRS_UBI_EXT=$(find "$EXTRACTION_DIR_"/UBIfs_extracted -type d | wc -l)
        print_output "[*] Extracted $ORANGE$FILES_UBI_EXT$NC files and $ORANGE$DIRS_UBI_EXT$NC directories from the firmware image via UBI deep extraction."
      fi
    done
 
    print_ln
    FILES_UBI_EXT=$(find "$EXTRACTION_DIR_" -type f | wc -l)
    DIRS_UBI_EXT=$(find "$EXTRACTION_DIR_" -type d | wc -l)
    print_output "[*] Extracted $ORANGE$FILES_UBI_EXT$NC files and $ORANGE$DIRS_UBI_EXT$NC directories from the firmware image."
    write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
    write_csv_log "UBI filesystem extractor" "$UBI_PATH_" "$EXTRACTION_DIR_" "$FILES_UBI_EXT" "$DIRS_UBI_EXT" "NA"
  else
    print_output "[-] First round UBI extractor failed!"
  fi
}

P16_EnGenius_decryptor.sh

用EnGenius提取固件镜像
engenius_enc_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
engenius_enc_extractor() {
  local ENGENIUS_ENC_PATH_="${1:-}"
  local EXTRACTION_FILE_="${2:-}"
 
  if ! [[ -f "$ENGENIUS_ENC_PATH_" ]]; then
    print_output "[-] No file for decryption provided"
    return
  fi
 
  sub_module_title "EnGenius encrypted firmware extractor"
 
  hexdump -C "$ENGENIUS_ENC_PATH_" | head | tee -a "$LOG_FILE" || true
 
  if [[ -f "$EXT_DIR"/engenius-decrypt.py ]]; then
 
      # 运行解密脚本
    python3 "$EXT_DIR"/engenius-decrypt.py "$ENGENIUS_ENC_PATH_" > "$EXTRACTION_FILE_"
  else
    print_output "[-] Decryptor not found - check your installation"
  fi
 
  print_ln
  if [[ -f "$EXTRACTION_FILE_" ]]; then
    print_output "[+] Decrypted EnGenius firmware file to $ORANGE$EXTRACTION_FILE_$NC"
    export FIRMWARE_PATH="$EXTRACTION_FILE_"
    MD5_DONE_DEEP+=( "$(md5sum "$ENGENIUS_ENC_PATH_" | awk '{print $1}')" )
    print_ln
    print_output "[*] Firmware file details: $ORANGE$(file "$EXTRACTION_FILE_")$NC"
    write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
    write_csv_log "EnGenius decryptor" "$ENGENIUS_ENC_PATH_" "$EXTRACTION_FILE_" "1" "NA" "NA"
    if [[ -z "${FW_VENDOR:-}" ]]; then
      FW_VENDOR="EnGenius"
      backup_var "FW_VENDOR" "$FW_VENDOR"
    fi
  else
    print_output "[-] Decryption of EnGenius firmware file failed"
  fi
}

P17_gpg_decompress.sh

提取没有加密的gpg压缩的固件镜像
gpg_decompress_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
gpg_decompress_extractor() {
  local GPG_FILE_PATH_="${1:-}"
  local EXTRACTION_FILE_="${2:-}"
 
  if ! [[ -f "$GPG_FILE_PATH_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
  sub_module_title "GPG compressed firmware extractor"
 
  # 列出包列表并解密   
  gpg --list-packets "$GPG_FILE_PATH_" 2>/dev/null | tee -a "$LOG_FILE"
  gpg --decrypt "$GPG_FILE_PATH_" > "$EXTRACTION_FILE_" || true
 
  print_ln
  if [[ -f "$EXTRACTION_FILE_" ]]; then
    print_output "[+] Extracted GPG compressed firmware file to $ORANGE$EXTRACTION_FILE_$NC"
    export FIRMWARE_PATH="$EXTRACTION_FILE_"
    backup_var "FIRMWARE_PATH" "$FIRMWARE_PATH"
    print_ln
    print_output "[*] Firmware file details: $ORANGE$(file "$EXTRACTION_FILE_")$NC"
    write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
    write_csv_log "GPG decompression" "$GPG_FILE_PATH_" "$EXTRACTION_FILE_" "1" "NA" "NA"
  else
    print_output "[-] Extraction of GPG compressed firmware file failed"
  fi
}

P18_qnap_decryptor.sh

从QNAP中提取加密固件映像
首先调用 qnap_enc_extractor 函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
qnap_enc_extractor() {
  local QNAP_ENC_PATH_="${1:-}"
  local EXTRACTION_FILE_="${2:-}"
  export QNAP=0
 
  if ! [[ -f "$QNAP_ENC_PATH_" ]]; then
    print_output "[-] No file for decryption provided"
    return
  fi
 
  sub_module_title "QNAP encrypted firmware extractor"
 
  hexdump -C "$QNAP_ENC_PATH_" | head | tee -a "$LOG_FILE" || true
 
  if [[ -f "$EXT_DIR"/PC1 ]]; then
    print_ln
    print_output "[*] Decrypting QNAP firmware with leaked key material ..."
    print_ln
 
      # 如果有PC1工具，则用PC1工具提取
    "$EXT_DIR"/PC1 d QNAPNASVERSION4 "$QNAP_ENC_PATH_" "$EXTRACTION_FILE_" | tee -a "$LOG_FILE"
  else
    print_output "[-] QNAP decryptor not found - check your installation"
  fi
 
  print_ln
  if [[ -f "$EXTRACTION_FILE_" && "$(file "$EXTRACTION_FILE_")" == *"gzip compressed data"* ]]; then
    print_output "[+] Decrypted QNAP firmware file to $ORANGE$EXTRACTION_FILE_$NC"
    MD5_DONE_DEEP+=( "$(md5sum "$QNAP_ENC_PATH_" | awk '{print $1}')" )
    export FIRMWARE_PATH="$EXTRACTION_FILE_"
    backup_var "FIRMWARE_PATH" "$FIRMWARE_PATH"
 
      # 如果提取出来是gzip文件，那再提取一次，使用下面的函数
    export QNAP=1
    print_ln
    print_output "[*] Firmware file details: $ORANGE$(file "$EXTRACTION_FILE_")$NC"
    print_ln
    write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
    write_csv_log "QNAP decryptor" "$QNAP_ENC_PATH_" "$EXTRACTION_FILE_" "1" "NA" "gzip compressed data"
    if [[ -z "${FW_VENDOR:-}" ]]; then
      FW_VENDOR="QNAP"
      backup_var "FW_VENDOR" "$FW_VENDOR"
    fi
 
  else
    print_output "[-] Decryption of QNAP firmware file failed"
  fi
}

qnap_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
qnap_extractor() {
  local DECRYPTED_FW_="${1:-}"
  if ! [[ -f "$DECRYPTED_FW_" ]]; then
    return
  fi
 
  sub_module_title "QNAP firmware extraction"
  print_output "[!] WARNING: This module is in an very early alpha state."
  print_output "[!] WARNING: Some areas of this module are not tested."
 
  # This module is a full copy of https://github.com/max-boehm/qnap-utils/blob/master/extract_qnap_fw.sh
  # some areas of this code are completely untested. Please report bugs via https://github.com/e-m-b-a/emba/issues
 
  QNAP_EXTRACTION_ROOT="$LOG_DIR"/firmware/qnap_extraction
  QNAP_EXTRACTION_ROOT_DST="$QNAP_EXTRACTION_ROOT"/root_filesystem
  mkdir -p "$QNAP_EXTRACTION_ROOT_DST" || true
 
  if file "$DECRYPTED_FW_" | grep -q ": gzip" ; then
    print_output "[*] Extracting $ORANGE$DECRYPTED_FW_$NC into $ORANGE$QNAP_EXTRACTION_ROOT$NC."
    mkdir -p "$QNAP_EXTRACTION_ROOT" || true
 
      # 解压gzip文件
    tar xvf "$DECRYPTED_FW_" -C "$QNAP_EXTRACTION_ROOT" 2>/dev/null || true | tee -a "$LOG_FILE"
    print_ln
    print_output "[*] Extracted firmware structure ($ORANGE$QNAP_EXTRACTION_ROOT$NC):"
 
      # 统计输出
        find "$QNAP_EXTRACTION_ROOT" -xdev -ls | tee -a "$LOG_FILE"
    print_files_dirs
    print_bar ""
  else
    print_output "[-] No QNAP firmware file found"
    return 1
  fi
 
  UIMAGE="$QNAP_EXTRACTION_ROOT/uImage"               # x31,x31+
  UBI="$QNAP_EXTRACTION_ROOT/rootfs2.ubi"             # x31,x31+
  IMAGE="$QNAP_EXTRACTION_ROOT_DST/image"
 
  # initial ramdisk root filesystem
  INITRAMFS="$QNAP_EXTRACTION_ROOT_DST/initramfs"        # x31,x31+
  INITRD="$QNAP_EXTRACTION_ROOT/initrd.boot"          # x10,x12,x19,x20,x21
  if [ ! -e "$INITRD" ]; then
    INITRD="$QNAP_EXTRACTION_ROOT/initrd"             # x51,x53
  fi
 
  ROOTFS2="$QNAP_EXTRACTION_ROOT/rootfs2.tgz"
  ROOTFS2_BZ="$QNAP_EXTRACTION_ROOT/rootfs2.bz"
  ROOTFS2_IMG="$QNAP_EXTRACTION_ROOT/rootfs2.img"
  ROOTFS_EXT="$QNAP_EXTRACTION_ROOT/rootfs_ext.tgz"
  QPKG="$QNAP_EXTRACTION_ROOT/qpkg.tar"
 
  if [ -e "$UBI" ]; then
    ROOTFS2="$QNAP_EXTRACTION_ROOT_DST/rootfs2.tgz"
    ROOTFS_EXT="$QNAP_EXTRACTION_ROOT_DST/rootfs_ext.tgz"
    QPKG="$QNAP_EXTRACTION_ROOT_DST/qpkg.tar"
  fi
 
  SYSROOT="$QNAP_EXTRACTION_ROOT_DST/sysroot"
  mkdir "$SYSROOT"
 
  if [ -e "$UIMAGE" ]; then
    print_ln
    print_output "[*] Scanning $ORANGE$UIMAGE$NC for (gzipped) parts..."
 
      # 识别是否为gzip文件，并定位到首次出现gzip文件头的偏移，用于后面的dd命令
    a=$(od -t x1 -w4 -Ad -v "$UIMAGE" | grep '1f 8b 08 00' | awk '{print $1}')
 
    if [ -n "$a" ]; then
 
        # 按照块大小（block size） $a 跳过第一个块，然后将其余的内容复制到输出文件（output file） $IMAGE.gz 中
      dd if="$UIMAGE" bs="$a" skip=1 of="$IMAGE.gz" status=none
 
        # 解压
      gunzip --quiet "$IMAGE.gz" || [ $? -eq 2 ]
      print_output "[+] Extracted and uncompressed $ORANGE$IMAGE$NC at offset $ORANGE$a$NC"
 
      i=0
      for a in $(od -t x1 -w4 -Ad -v "$IMAGE" | grep '1f 8b 08 00' | awk '{print $1}'); do
        i=$((i+1))
 
          # 再识别IMAGE文件里的所有gzip文件，分别提取所有部分
        dd if="$IMAGE" bs="$a" skip=1 of="$IMAGE.part$i.gz" status=none
        gunzip --quiet "$IMAGE.part$i.gz" || [ $? -eq 2 ]
        print_output "[+] Extracted and uncompressed '$IMAGE.part$i' at offset $a"
      done
 
      if [ $i -gt 0 ]; then
        mv "$IMAGE.part$i" "$INITRAMFS" || true
        print_output "[*] Renamed $ORANGE$IMAGE.part$i$NC to $ORANGE$INITRAMFS$NC"
        rm "$IMAGE" || true
      fi
    fi
    print_files_dirs
    print_bar ""
  fi
 
  if [ -e "$UBI" ]; then
      # rootfs2.ubi
    print_ln
    print_output "[*] Unpacking $ORANGE$UBI$NC."
    # TODO: we should evaluate moving to the EMBA UBI extractor in the future
 
    # see http://trac.gateworks.com/wiki/linux/ubi
    #
    # apt-get install mtd-utils
 
    # 256MB flash
    modprobe -r nandsim || true
    if [ -e /dev/mtdblock0 ]; then
      print_output "[-] /dev/mtdblock0 does already exist! Exiting to not overwrite it."; exit
    fi
    modprobe nandsim first_id_byte=0x2c second_id_byte=0xda third_id_byte=0x90 fourth_id_byte=0x95
 
    print_output "[*] Copy UBI image into simulated flash device"
    # populate NAND with an existing ubi:
    modprobe mtdblock
    dd if="$UBI" of=/dev/mtdblock0 bs=2048 status=none
 
    print_output "[*] Attach simulated flash device"
    # attach ubi
    modprobe ubi
    ubiattach /dev/ubi_ctrl -m0 -O2048
    # ubinfo -a
 
    print_output "[*] Mounting ubifs file system"
    # mount the ubifs to host
    modprobe ubifs
    local TMP_EXT_MOUNT="$TMP_DIR""/ext_mount_$RANDOM"
    mkdir -p "$TMP_EXT_MOUNT" || true
    mount -t ubifs ubi0 "$TMP_EXT_MOUNT"
    if mount | grep -q ext_mount; then
      print_output "[*] Copying contents from UBI mount"
      cp -a "$TMP_EXT_MOUNT"/boot/* "$QNAP_EXTRACTION_ROOT_DST" || true
      print_ln
      print_output "[*] Extracted firmware structure ($ORANGE$QNAP_EXTRACTION_ROOT_DST$NC):"
      find "$QNAP_EXTRACTION_ROOT_DST" -xdev -ls | tee -a "$LOG_FILE"
 
      print_output "[*] UBI cleanup"
      umount "$TMP_EXT_MOUNT"
    else
      print_output "[-] Something went wrong!"
    fi
    rm -r "$TMP_EXT_MOUNT" || true
    ubidetach /dev/ubi_ctrl -m0
    modprobe -r nandsim
    print_files_dirs
    print_bar ""
  fi
 
    # 提取初始内存文件系统
  if [ -e "$INITRAMFS" ]; then
    print_ln
    print_output "[*] Extracting $ORANGE$INITRAMFS$NC."
    (cd "$SYSROOT" && (cpio -i --make-directories||true) ) < "$INITRAMFS"
    print_ln
    print_output "[*] Extracted firmware structure ($ORANGE$SYSROOT$NC):"
    find "$SYSROOT" -xdev -ls | tee -a "$LOG_FILE"
    print_files_dirs
    print_bar ""
  fi
 
  if [ -e "$INITRD" ]; then
    print_ln
 
    # 如果是lzma压缩文件，则解压
    if file "$INITRD" | grep -q LZMA ; then
      print_output "[*] Extracting $ORANGE$INITRD$NC (LZMA)."
      lzma -d <"$INITRD" | (cd "$SYSROOT" && (cpio -i --make-directories||true) )
      print_ln
      print_output "[*] Extracted firmware structure ($ORANGE$SYSROOT$NC):"
      find "$SYSROOT" -xdev -ls | tee -a "$LOG_FILE"
      print_files_dirs
      print_bar ""
    fi
 
      # 如果是gzip，则解压
    if file "$INITRD" | grep -q gzip ; then
      print_ln
      print_output "[*] Extracting $ORANGE$INITRD$NC (gzip)."
      gzip -d <"$INITRD" >"$QNAP_EXTRACTION_ROOT_DST/initrd.$$"
      print_output "[*] Mounting $ORANGE$INITRD$NC."
      local TMP_EXT_MOUNT="$TMP_DIR""/ext_mount_$RANDOM"
      mkdir -p "$TMP_EXT_MOUNT" || true
      mount -t ext2 "$QNAP_EXTRACTION_ROOT_DST/initrd.$$" "$TMP_EXT_MOUNT" -oro,loop
      if mount | grep -q ext_mount; then
        cp -a "$TMP_EXT_MOUNT"/* "$SYSROOT" || true
        umount "$TMP_EXT_MOUNT"
        print_ln
        print_output "[*] Extracted firmware structure ($ORANGE$SYSROOT$NC):"
        find "$SYSROOT" -xdev -ls | tee -a "$LOG_FILE"
        rm "$QNAP_EXTRACTION_ROOT_DST/initrd.$$" || true
      else
        print_output "[-] Something went wrong!"
      fi
      rm -r "$TMP_EXT_MOUNT" || true
      print_files_dirs
      print_bar ""
    fi
  fi
 
    # 用tar解压
  if [ -e "$ROOTFS2" ]; then
    print_ln
    print_output "[*] Extracting $ORANGE$ROOTFS2$NC (gzip, tar)."
    tar -xvzf "$ROOTFS2" -C "$SYSROOT"
    print_ln
    print_output "[*] Extracted firmware structure ($ORANGE$SYSROOT$NC):"
    find "$SYSROOT" -xdev -ls | tee -a "$LOG_FILE"
    print_files_dirs
    print_bar ""
  fi
 
    # 用lzma解压
  if [ -e "$ROOTFS2_BZ" ]; then
    print_ln
    if file "$ROOTFS2_BZ" | grep -q "LZMA"; then
      print_output "[*] Extracting $ORANGE$ROOTFS2_BZ$NC (LZMA)."
      lzma -d <"$ROOTFS2_BZ" | (cd "$SYSROOT" && (cpio -i --make-directories||true) )
    else
      print_output "[*] Extracting $ORANGE$ROOTFS2_BZ$NC (bzip2, tar)."
      tar -xvjf "$ROOTFS2_BZ" -C "$SYSROOT"
    fi
    print_ln
    print_output "[*] Extracted firmware structure ($ORANGE$SYSROOT$NC):"
    find "$SYSROOT" -xdev -ls | tee -a "$LOG_FILE"
    print_files_dirs
    print_bar ""
  fi
 
  if [ -f "$ROOTFS2_IMG" ]; then
    print_ln
    print_output "[*] Extracting $ORANGE$ROOTFS2_IMG$NC (ext2)..."
    local TMP_EXT_MOUNT="$TMP_DIR""/ext_mount_$RANDOM"
    mkdir -p "$TMP_EXT_MOUNT" || true
    mount -t ext2 "$ROOTFS2_IMG" "$TMP_EXT_MOUNT" -oro,loop
    if mount | grep -q ext_mount; then
 
        # 挂载ROOTFS2_IMG，然后解压里面的rootfs2.bz
      tar -xvjf "$TMP_EXT_MOUNT"/rootfs2.bz -C "$SYSROOT"
      print_ln
      print_output "[*] Extracted firmware structure ($ORANGE$SYSROOT$NC):"
      find "$SYSROOT" -xdev -ls | tee -a "$LOG_FILE"
      umount "$TMP_EXT_MOUNT"
    else
      print_output "[-] Something went wrong!"
    fi
    rm -r "$TMP_EXT_MOUNT" || true
    print_files_dirs
    print_bar ""
  fi
 
  if [ -e "$ROOTFS_EXT" ]; then
    print_ln
    print_output "[*] Extracting EXT filesystem $ORANGE$ROOTFS_EXT$NC."
 
      # tar解压ext根文件系统
    tar xzvf "$ROOTFS_EXT" -C "$QNAP_EXTRACTION_ROOT_DST"
    print_output "[*] Mounting EXT filesystem $ORANGE$ROOTFS_EXT$NC."
    local TMP_EXT_MOUNT="$TMP_DIR""/ext_mount_$RANDOM"
    mkdir -p "$TMP_EXT_MOUNT" || true
 
      # 挂载ext根文件系统，然后复制出根文件系统下的所有文件
    mount "$QNAP_EXTRACTION_ROOT_DST"/rootfs_ext.img "$TMP_EXT_MOUNT" -oro,loop
    if mount | grep -q ext_mount; then
      cp -a "$TMP_EXT_MOUNT"/* "$SYSROOT" || true
      umount "$TMP_EXT_MOUNT"
    fi
    print_output "[*] Removing EXT filesystem ${ORANGE}rootfs_ext.img$NC."
    rm "$QNAP_EXTRACTION_ROOT_DST"/rootfs_ext.img || true
    rm -r "$TMP_EXT_MOUNT" || true
    print_ln
    print_output "[*] Extracted firmware structure ($ORANGE$SYSROOT$NC):"
    find "$SYSROOT" -xdev -ls | tee -a "$LOG_FILE"
    print_files_dirs
    print_bar ""
  fi
 
    # 查找根文件系统/opt/source下的所有tgz文件并解压到/usr/local下
  USR_LOCAL=$(find "$SYSROOT/opt/source" -name "*.tgz" 2>/dev/null)
  # if [[ "${#USR_LOCAL[@]}" -gt 0 ]]; then
  if [[ -v USR_LOCAL[@] ]]; then
    print_ln
    for f in "${USR_LOCAL[@]}"; do
      print_output "[*] Extracting $ORANGE$f$NC -> ${ORANGE}sysroot/usr/local$NC ..."
      mkdir -p "$SYSROOT/usr/local" || true
      tar xvzf "$f" -C "$SYSROOT/usr/local"
    done
    print_files_dirs
    print_bar ""
  fi
 
    # "$QNAP_EXTRACTION_ROOT/qpkg.tar"
  if [ -e "$QPKG" ]; then
    print_ln
    print_output "[*] Extracting $ORANGE$QPKG$NC."
    mkdir -p "$QNAP_EXTRACTION_ROOT_DST/qpkg" || true
 
      # 解压
    tar xvf "$QPKG" -C "$QNAP_EXTRACTION_ROOT_DST/qpkg"
      # 从解压出来的文件夹里继续搜tgz解压
    for f in "$QNAP_EXTRACTION_ROOT_DST"/qpkg/*.tgz; do
      if file "$f" | grep -q gzip; then
        print_output "[*] Extracting QPKG $ORANGE$f$NC."
        tar tvzf "$f" > "$f".txt
      fi
    done
    print_files_dirs
    print_bar ""
  fi
 
    # 解压出一些服务的tgz
  for name in apache_php5 mysql5 mariadb5; do
    if [ -e "$QNAP_EXTRACTION_ROOT_DST/qpkg/$name.tgz" ]; then
      print_output "[*] Extracting ${ORANGE}qpkg/$name.tgz$NC -> ${ORANGE}sysroot/usr/local$NC ..."
      tar xvzf "$QNAP_EXTRACTION_ROOT_DST/qpkg/$name.tgz" -C "$SYSROOT/usr/local"
    fi
  done
 
    # Boost C++库解压到$SYSROOT/usr/lib下
  if [ -e "$QNAP_EXTRACTION_ROOT_DST"/qpkg/libboost.tgz ]; then
    print_ln
    print_output "[*] Extracting ${ORANGE}qpkg/libboost.tgz$NC -> ${ORANGE}sysroot/usr/lib$NC."
    mkdir -p "$SYSROOT/usr/lib" || true
    tar xvzf "$QNAP_EXTRACTION_ROOT_DST"/qpkg/libboost.tgz -C "$SYSROOT/usr/lib"
  elif [ -e "$QNAP_EXTRACTION_ROOT_DST"/qpkg/DSv3.tgz ]; then
    print_output "[*] Extracting ${ORANGE}libboost$NC from ${ORANGE}qpkg/DSv3.tgz$NC -> ${ORANGE}sysroot/usr/lib$NC."
    tar tzf "$QNAP_EXTRACTION_ROOT_DST"/qpkg/DSv3.tgz |grep libboost | tar xzf "$QNAP_EXTRACTION_ROOT_DST"/qpkg/DSv3.tgz -C "$SYSROOT" -T -
  fi
 
  if [[ -d "$SYSROOT"/usr/lib ]]; then
    HOME_DIR="$(pwd)"
 
      # 创建软链接
    (cd "$SYSROOT/usr/lib" || exit; for f in libboost*.so.1.42.0; do ln -s "$f" "${f%.1.42.0}"; done)
    cd "$HOME_DIR" || exit
  fi
  print_files_dirs
  print_bar ""
}

P19_bsd_ufs_mounter.sh

挂载和提取BSD UFS映像
ufs_extractor函数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
ufs_extractor() {
  local UFS_PATH_="${1:-}"
  local EXTRACTION_DIR_="${2:-}"
  local TMP_UFS_MOUNT="$TMP_DIR""/ufs_mount_$RANDOM"
  local DIRS_UFS_MOUNT=0
  FILES_UFS_MOUNT=0
 
  if ! [[ -f "$UFS_PATH_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
  sub_module_title "UFS filesystem extractor"
 
  mkdir -p "$TMP_UFS_MOUNT" 2>/dev/null || true
  print_output "[*] Trying to mount $ORANGE$UFS_PATH_$NC to $ORANGE$TMP_UFS_MOUNT$NC directory"
    # 装载ufs内核模块
    modprobe ufs
    # 挂载ufs镜像
  mount -r -t ufs -o ufstype=ufs2 "$UFS_PATH_" "$TMP_UFS_MOUNT"
 
  if mount | grep -q ufs_mount; then
    print_output "[*] Copying $ORANGE$TMP_UFS_MOUNT$NC to firmware tmp directory ($ORANGE$EXTRACTION_DIR_$NC)"
    mkdir -p "$EXTRACTION_DIR_" 2>/dev/null || true
 
      # 把ufs里所有文件拷贝出来
    cp -pri "$TMP_UFS_MOUNT"/* "$EXTRACTION_DIR_" 2>/dev/null || true
    print_ln
 
      # 统计文件
    print_output "[*] Using the following firmware directory ($ORANGE$EXTRACTION_DIR_$NC) as base directory:"
    find "$EXTRACTION_DIR_" -xdev -maxdepth 1 -ls | tee -a "$LOG_FILE"
    print_ln
    print_output "[*] Unmounting $ORANGE$TMP_UFS_MOUNT$NC directory"
 
    FILES_UFS_MOUNT=$(find "$EXTRACTION_DIR_" -type f | wc -l)
    DIRS_UFS_MOUNT=$(find "$EXTRACTION_DIR_" -type d | wc -l)
    print_output "[*] Extracted $ORANGE$FILES_UFS_MOUNT$NC files and $ORANGE$DIRS_UFS_MOUNT$NC directories from the firmware image."
    write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
    write_csv_log "UFS filesystem extractor" "$UFS_PATH_" "$EXTRACTION_DIR_" "$FILES_UFS_MOUNT" "$DIRS_UFS_MOUNT" "NA"
    umount "$TMP_UFS_MOUNT" 2>/dev/null || true
  fi
  rm -r "$TMP_UFS_MOUNT"
}

P20_foscam_decryptor.sh

提取Foscam公司的加密固件映像
首先调用foscam_enc_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
foscam_enc_extractor() {
  local FOSCAM_ENC_PATH_="${1:-}"
  local EXTRACTION_FILE_="${2:-}"
  local FOSCAM_FILE_CHECK=""
  local KEY_FILE="$CONFIG_DIR/foscam_enc_keys.txt"
 
  if ! [[ -f "$FOSCAM_ENC_PATH_" ]]; then
    print_output "[-] No file for decryption provided"
    return
  fi
 
    # config文件夹下的foscam_enc_keys.txt保存密钥列表
  if ! [[ -f "$KEY_FILE" ]]; then
    print_output "[-] No key file found in config directory"
    return
  fi
 
  sub_module_title "Foscam encrypted firmware extractor"
 
  hexdump -C "$FOSCAM_ENC_PATH_" | head | tee -a "$LOG_FILE" || true
 
    # 保存密钥列表
  mapfile FOSCAM_KEYS < <(grep -v "ID" "$KEY_FILE" | cut -d\; -f2 | tr -d \')
  for _FOSCAM_KEY in "${FOSCAM_KEYS[@]}"; do
    FOSCAM_DECRYTED=0
    print_output "[*] Testing FOSCAM decryption key $ORANGE$_FOSCAM_KEY$NC."
 
    # shellcheck disable=SC2086
      # aes解密
    openssl enc -d -aes-128-cbc -md md5 -k $_FOSCAM_KEY -in "$FOSCAM_ENC_PATH_" > "$EXTRACTION_FILE_" || true
 
    if [[ -f "$EXTRACTION_FILE_" ]]; then
 
        # 识别解密后的文件类型
      FOSCAM_FILE_CHECK=$(file "$EXTRACTION_FILE_")
      if [[ "$FOSCAM_FILE_CHECK" =~ .*gzip\ compressed\ data.* ]]; then
 
        # 如果是gzip文件，那就到后面进行ubi提取
        print_ln
        print_output "[+] Decrypted Foscam firmware file to $ORANGE$EXTRACTION_FILE_$NC"
        export FIRMWARE_PATH="$EXTRACTION_FILE_"
        backup_var "FIRMWARE_PATH" "$FIRMWARE_PATH"
        print_ln
        print_output "[*] Firmware file details: $ORANGE$(file "$EXTRACTION_FILE_")$NC"
        write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
        write_csv_log "Foscam decryptor" "$FOSCAM_ENC_PATH_" "$EXTRACTION_FILE_" "1" "NA" "NA"
        FOSCAM_DECRYTED=1
        if [[ -z "${FW_VENDOR:-}" ]]; then
          FW_VENDOR="Foscam"
        fi
 
        MD5_DONE_DEEP+=( "$(md5sum "$FOSCAM_ENC_PATH_" | awk '{print $1}')" )
        foscam_ubi_extractor "$EXTRACTION_FILE_"
        # as we have already found a working key we can now exit the loop
        break
      fi
    fi
  done
 
  if [[ "$FOSCAM_DECRYTED" -ne 1 ]]; then
    print_output "[-] Decryption of Foscam firmware file failed"
  fi
}

foscam_ubi_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
# TODO: Check if we can improve our ubifs extractor in a way to support this Foscam thing
# without the following function! Currently it is working together with our
# deep extractor quite fine.
foscam_ubi_extractor() {
  local FIRMWARE_PATH_="${1:-}"
  local MTD_DEVICE=""
  local UBI_MNT_PT="$LOG_DIR"/tmp/ubi_mnt_foscam
  local EXTRACTION_DIR_="$LOG_DIR/firmware/foscam_ubi_extractor"
  local EXTRACTION_DIR_GZ="$LOG_DIR/firmware/foscam_gz_extractor"
  local UBI_DEV=""
  local UBI_DEVS=()
  FOSCAM_UBI_FILES=0
  FOSCAM_UBI_DIRS=0
 
  if ! [[ -f "$FIRMWARE_PATH_" ]]; then
    print_output "[-] No file for extraction found"
    return
  fi
 
  print_output "[*] Extracting decrypted firmware to $ORANGE$EXTRACTION_DIR_GZ$NC"
  mkdir -p "$EXTRACTION_DIR_GZ" || true
 
  # 创建文件夹，然后解压到文件夹内
  tar -xzf "$FIRMWARE_PATH_" -C "$EXTRACTION_DIR_GZ" || true
 
  # check if we have the kernel modules available - special interest in docker
  if ! [[ -d "/lib/modules/" ]]; then
    print_output "[-] Kernel modules not mounted from host system - please update your docker-compose file!"
    return
  fi
 
  if [[ -f "$EXTRACTION_DIR_GZ"/app_ubifs ]]; then
    print_output "[*] 2nd extraction round successful - ${ORANGE}app_ubifs$NC found"
    print_output "[*] Loading nandsim kernel module"
    if lsmod | grep -q "^nandsim[[:space:]]"; then
      # we need to load nandsim with some parameters - unload it before
      modprobe -r nandsim
    fi
      # 像前面的模块一样加载内核模块
    modprobe nandsim first_id_byte=0x2c second_id_byte=0xac third_id_byte=0x90 fourth_id_byte=0x15
    MTD_DEVICE=$(grep "mtd[0-9]" /proc/mtd | cut -d: -f1)
    print_output "[*] Found $ORANGE/dev/$MTD_DEVICE$NC MTD device"
    print_output "[*] Erasing $ORANGE/dev/$MTD_DEVICE$NC MTD device"
    flash_erase /dev/"$MTD_DEVICE" 0 0 || true
    print_output "[*] Formating $ORANGE/dev/$MTD_DEVICE$NC MTD device"
    ubiformat /dev/"$MTD_DEVICE" -O 2048 -f "$EXTRACTION_DIR_GZ"/app_ubifs || true
    if ! lsmod | grep -q "^ubi[[:space:]]"; then
      print_output "[*] Loading ubi kernel module"
      modprobe ubi
    fi
    print_output "[*] Attaching ubi device"
    ubiattach -p /dev/"$MTD_DEVICE" -O 2048
 
    # should be only one UBI dev, but just in case ...
    mapfile -t UBI_DEVS < <(find /dev -iname "ubi[0-9]_[0-9]")
    for UBI_DEV in "${UBI_DEVS[@]}"; do
      local UBI_MNT_PT="$UBI_MNT_PT-$RANDOM"
      print_output "[*] Mounting $ORANGE$UBI_DEV$NC ubi device to $ORANGE$UBI_MNT_PT$NC"
      mkdir -p "$UBI_MNT_PT" || true
 
        # 挂载ubi设备
      mount -t ubifs "$UBI_DEV" "$UBI_MNT_PT"
      print_output "[*] Copy mounted ubi device to $ORANGE$EXTRACTION_DIR_/$UBI_DEV$NC"
      mkdir -p "$EXTRACTION_DIR_/$UBI_DEV"
 
        # 复制文件，然后取消挂载、删除。
      cp -pri "$UBI_MNT_PT" "$EXTRACTION_DIR_/$UBI_DEV"
      umount "$UBI_MNT_PT" || true
      rm -r "$UBI_MNT_PT" || true
    done
 
      # 取消附加、解除内核模块加载
    # do some cleanup
    print_output "[*] Detaching ubi device"
    ubidetach -d 0 || true
    print_output "[*] Unloading nandsim module"
    modprobe -r nandsim || true
    print_output "[*] Unloading ubi module"
    modprobe -r ubi || true
 
    if [[ -d "$EXTRACTION_DIR_" ]]; then
      FOSCAM_UBI_FILES=$(find "$EXTRACTION_DIR_" -type f | wc -l)
      FOSCAM_UBI_DIRS=$(find "$EXTRACTION_DIR_" -type d | wc -l)
    fi
 
    if [[ "$FOSCAM_UBI_FILES" -gt 0 ]]; then
      print_ln
      print_output "[*] Extracted $ORANGE$FOSCAM_UBI_FILES$NC files and $ORANGE$FOSCAM_UBI_DIRS$NC directories from the firmware image."
      write_csv_log "Foscam UBI extractor" "$FIRMWARE_PATH_" "$EXTRACTION_DIR_" "$FOSCAM_UBI_FILES" "$FOSCAM_UBI_DIRS" "NA"
      export FIRMWARE_PATH="$LOG_DIR"/firmware
      backup_var "FIRMWARE_PATH" "$FIRMWARE_PATH"
      write_csv_log "Foscam decryptor/extractor" "$FIRMWARE_PATH_" "$EXTRACTION_DIR_" "$FOSCAM_UBI_FILES" "$FOSCAM_UBI_DIRS" "NA"
    fi
  fi
}

P21_buffalo_decryptor.sh

提取Buffalo公司的加密固件映像
buffalo_enc_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
buffalo_enc_extractor() {
  local BUFFALO_ENC_PATH_="${1:-}"
  local EXTRACTION_FILE_="${2:-}"
  local BUFFALO_FILE_CHECK=""
 
  if ! [[ -f "$BUFFALO_ENC_PATH_" ]]; then
    print_output "[-] No file for decryption provided"
    return
  fi
 
  sub_module_title "Buffalo encrypted firmware extractor"
 
  hexdump -C "$BUFFALO_ENC_PATH_" | head | tee -a "$LOG_FILE" || true
  print_ln
 
  BUFFALO_DECRYTED=0
  local BUFFALO_ENC_PATH_STRIPPED
  BUFFALO_ENC_PATH_STRIPPED="$LOG_DIR/firmware/$(basename "$BUFFALO_ENC_PATH_").stripped"
 
  print_output "[*] Removing initial 208 bytes from header to prepare firmware for decryption"
 
    # 去除前面多余的头
  dd bs=208 skip=1 if="$BUFFALO_ENC_PATH_" of="$BUFFALO_ENC_PATH_STRIPPED"
  hexdump -C "$BUFFALO_ENC_PATH_STRIPPED" | head | tee -a "$LOG_FILE" || true
  print_ln
 
  print_output "[*] Decrypting firmware ..."
 
  # 使用扩展下的解密工具
  "$EXT_DIR"/buffalo-enc.elf -d -i "$BUFFALO_ENC_PATH_STRIPPED" -o "$EXTRACTION_FILE_"
  hexdump -C "$EXTRACTION_FILE_" | head | tee -a "$LOG_FILE" || true
  print_ln
 
  if [[ -f "$EXTRACTION_FILE_" ]]; then
 
      # 检查文件类型，是不是要的文件
    BUFFALO_FILE_CHECK=$(file "$EXTRACTION_FILE_")
    if [[ "$BUFFALO_FILE_CHECK" =~ .*u-boot\ legacy\ uImage,\ .* ]]; then
      print_ln
      print_output "[+] Decrypted Buffalo firmware file to $ORANGE$EXTRACTION_FILE_$NC"
      MD5_DONE_DEEP+=( "$(md5sum "$BUFFALO_ENC_PATH_" | awk '{print $1}')" )
      export FIRMWARE_PATH="$EXTRACTION_FILE_"
      backup_var "FIRMWARE_PATH" "$FIRMWARE_PATH"
      print_ln
      print_output "[*] Firmware file details: $ORANGE$(file "$EXTRACTION_FILE_")$NC"
      write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
      write_csv_log "Buffalo decryptor" "$BUFFALO_ENC_PATH_" "$EXTRACTION_FILE_" "1" "NA" "NA"
      BUFFALO_DECRYTED=1
      if [[ -z "${FW_VENDOR:-}" ]]; then
        FW_VENDOR="BUFFALO"
      fi
    fi
  fi
 
  if [[ "$BUFFALO_DECRYTED" -ne 1 ]]; then
    print_output "[-] Decryption of Buffalo firmware file failed"
  fi
}

P22_Zyxel_zip_decrypt.sh

提取被密码保护的Zyxel固件镜像
zyxel_zip_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
zyxel_zip_extractor() {
  local RI_FILE_="${1:-}"
  local EXTRACTION_DIR_="${2:-}"
 
  local RI_FILE_BIN=""
  local ZLD_DIR=""
  local RI_FILE_BIN_PATH=""
  local ZLD_BINS=()
  local ZLD_BIN=""
 
  sub_module_title "Zyxel protected ZIP firmware extractor"
 
  if ! [[ -f "$RI_FILE_" ]]; then
    print_output "[-] Zyxel - No file for extraction provided"
    return
  fi
  if ! [[ "$RI_FILE_" =~ .*\.ri ]]; then
    print_output "[-] Zyxel - No valid ri file for extraction provided"
    return
  fi
 
    # 直接用binwalk深度提取，下面给出此函数分析
  binwalk_deep_extract_helper 1 "$RI_FILE_" "$EXTRACTION_DIR_"
  print_ln
 
  # 如果有这个命令
  if command -v jchroot > /dev/null; then
    CHROOT="jchroot"
    # OPTS see https://github.com/vincentbernat/jchroot#security-note
    OPTS=(-n emba -U -u 0 -g 0 -M "0 $(id -u) 1" -G "0 $(id -g) 1")
    print_output "[*] Using ${ORANGE}jchroot${NC} for building more secure chroot environments"
  else
    print_output "[-] No jchroot binary found ..."
    return
  fi
 
    # 在提取出的文件夹里找zld_fsextract的文件
  mapfile -t ZLD_BINS < <(find "$EXTRACTION_DIR_" -name "zld_fsextract")
 
    # 去掉.ri后缀然后接上.bin后缀
  RI_FILE_BIN="$(basename -s .ri "$RI_FILE_")".bin
 
  for ZLD_BIN in "${ZLD_BINS[@]}"; do
    local FILES_ZYXEL=0
    local DIRS_ZYXEL=0
    print_output "[*] Checking $ORANGE$ZLD_BIN$NC"
 
      # 获取文件夹名
    ZLD_DIR=$(dirname "$ZLD_BIN")
 
      # 找到第一个名字叫这个的文件
    RI_FILE_BIN_PATH=$(find "$LOG_DIR"/firmware -name "$RI_FILE_BIN" | head -1)
    # => this should be the protected Zip file
 
      # 压缩包是一个加密压缩包，但是密钥肯定在程序执行时会出现在内存中
    if [[ $(file "$ZLD_BIN") == *"ELF"* ]] && [[ $(file "$RI_FILE_BIN_PATH") == *"Zip archive data"* ]]; then
      print_output "[*] Found Zyxel environment in $ORANGE$ZLD_DIR$NC"
      # now we know that we have an elf for extraction and and unzip binary in the extraction dir
      # this is everything we need for the key
      if ( file "$ZLD_BIN" | grep -q "ELF 32-bit MSB executable, MIPS, N32 MIPS64 rel2 version 1" ) ; then
        # todo: check if Zyxel also uses other architectures
        local EMULATOR="qemu-mipsn32-static"
        print_output "[*] Found valid emulator $ORANGE$EMULATOR$NC"
      else
        print_output "[-] WARNING: Unsupported architecture for key identification:"
        print_output "$(indent "$(file "$ZLD_BIN")")"
        print_output "[-] Please open an issue at https://github.com/e-m-b-a/emba/issues"
        continue
      fi
 
      print_output "[*] Running Zyxel emulation for key extraction ..."
 
      if ! [[ -e "$(command -v "$EMULATOR")" ]]; then
        print_output "[-] No valid emulator ($ORANGE$EMULATOR$NC) found in your environment"
        return
      fi
 
      cp "$(command -v "$EMULATOR")" "$ZLD_DIR" || ( print_output "[-] Something went wrong" && return)
      cp "$RI_FILE_BIN_PATH" "$ZLD_DIR" || ( print_output "[-] Something went wrong" && return)
      ZLD_BIN=$(basename "$ZLD_BIN")
 
        # 模拟执行
      timeout --preserve-status --signal SIGINT 2s "$CHROOT" "${OPTS[@]}" "$ZLD_DIR" -- ./"$EMULATOR" -strace ./"$ZLD_BIN" "$RI_FILE_BIN" AABBCCDD >> "$LOG_PATH_MODULE"/zld_strace.log 2>&1 || true
 
        # 删除模拟器
      rm "$ZLD_DIR"/"$EMULATOR" || true
 
      if [[ -f "$LOG_PATH_MODULE"/zld_strace.log ]] && [[ -s "$LOG_PATH_MODULE"/zld_strace.log ]]; then
 
          # 在模拟执行的跟踪日志中匹配密钥
        ZIP_KEY=$(grep -a -E "^[0-9]+\ execve.*AABBCCDD\",\"-o" "$LOG_PATH_MODULE"/zld_strace.log| cut -d, -f6 | sort -u | sed 's/^\"//' | sed 's/\"$//')
      else
        print_output "[-] No qemu strace log generated -> no further processing possible"
      fi
 
        # 如果能找到密钥
      # if we have found a ZIP_KEY:
      if [[ -v ZIP_KEY ]]; then
        print_ln
        print_output "[+] Possible ZIP key detected: $ORANGE$ZIP_KEY$NC" "" "$LOG_PATH_MODULE/zld_strace.log"
 
          # 使用密钥解压
        7z x -p"$ZIP_KEY" -o"$EXTRACTION_DIR_"/firmware_zyxel_extracted "$RI_FILE_BIN_PATH" || true
 
        FILES_ZYXEL=$(find "$EXTRACTION_DIR_"/firmware_zyxel_extracted -type f | wc -l)
        DIRS_ZYXEL=$(find "$EXTRACTION_DIR_"/firmware_zyxel_extracted -type d | wc -l)
 
        print_ln
        print_output "[*] Zyxel 1st stage - Extracted $ORANGE$FILES_ZYXEL$NC files and $ORANGE$DIRS_ZYXEL$NC directories from the firmware image."
        write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
        write_csv_log "Zyxel extractor" "$RI_FILE_BIN_PATH" "$EXTRACTION_DIR_/firmware_zyxel_extracted" "$FILES_ZYXEL" "$DIRS_ZYXEL" "NA"
      else
        print_output "[-] No ZIP key detected -> no further processing possible"
      fi
 
      # if it was possible to extract something with the key:
      if [[ "$FILES_ZYXEL" -gt 0 ]]; then
        # compress.img ist the firmware -> letz search for it
        COMPRESS_IMG=$(find "$EXTRACTION_DIR_"/firmware_zyxel_extracted -type f -name compress.img | sort -u)
        if [[ $(file "$COMPRESS_IMG") == *"Squashfs"* ]]; then
          print_output "[+] Found valid ${ORANGE}compress.img$GREEN and extract it now"
          binwalk_deep_extract_helper 1 "$COMPRESS_IMG" "$EXTRACTION_DIR_/firmware_zyxel_extracted/compress_img_extracted"
          FILES_ZYXEL=$(find "$EXTRACTION_DIR_"/firmware_zyxel_extracted/compress_img_extracted -type f | wc -l)
          DIRS_ZYXEL=$(find "$EXTRACTION_DIR_"/firmware_zyxel_extracted/compress_img_extracted -type d | wc -l)
          print_output "[*] Zyxel 2nd stage - Extracted $ORANGE$FILES_ZYXEL$NC files and $ORANGE$DIRS_ZYXEL$NC directories from the firmware image."
          write_csv_log "Zyxel extractor" "$RI_FILE_BIN_PATH" "$EXTRACTION_DIR_/firmware_zyxel_extracted/compress_img_extracted" "$FILES_ZYXEL" "$DIRS_ZYXEL" "NA"
          export FIRMWARE_PATH="$LOG_DIR"/firmware/
          backup_var "FIRMWARE_PATH" "$FIRMWARE_PATH"
          print_ln
          break
        else
          print_output "[-] No valid ${ORANGE}compress.img$NC file found"
        fi
      else
        print_output "[-] 1st stage Zip extraction failed"
      fi
      print_ln
    else
      print_output "[-] No environment for Zyxel decryption found"
    fi
  done
}

binwalk_deep_extract_helper函数（在P60_firmware_bin_extractor.sh文件内定义）：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
binwalk_deep_extract_helper() {
  # Matryoshka mode is first parameter: 1 - enable, 0 - disable
  local MATRYOSHKA_="${1:-0}"
  local FILE_TO_EXTRACT_="${2:-}"
  local DEST_FILE_="${3:-}"
 
  if ! [[ -f "$FILE_TO_EXTRACT_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
  if [[ "$BINWALK_VER_CHECK" == 1 ]]; then
 
      # 如果参数是1，则递归提取
    if [[ "$MATRYOSHKA_" -eq 1 ]]; then
 
        # -M：Recursively scan extracted files（递归扫描提取的文件）
      binwalk --run-as=root --preserve-symlinks --dd='.*' -e -M -C "$DEST_FILE_" "$FILE_TO_EXTRACT_" | tee -a "$LOG_FILE" || true
    else
      # no more Matryoshka mode ... we are doing it manually and check the files every round via MD5
      binwalk --run-as=root --preserve-symlinks --dd='.*' -e -C "$DEST_FILE_" "$FILE_TO_EXTRACT_" | tee -a "$LOG_FILE" || true
    fi
  else
    if [[ "$MATRYOSHKA_" -eq 1 ]]; then
      binwalk --dd='.*' -e -M -C "$DEST_FILE_" "$FILE_TO_EXTRACT_" | tee -a "$LOG_FILE" || true
    else
      binwalk --dd='.*' -e -C "$DEST_FILE_" "$FILE_TO_EXTRACT_" | tee -a "$LOG_FILE" || true
    fi
  fi
}

P23_qemu_qcow_mounter.sh

挂载和提取Qemu QCOW2镜像
qcow_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
qcow_extractor() {
  local QCOW_PATH_="${1:-}"
  local EXTRACTION_DIR_="${2:-}"
  local TMP_QCOW_MOUNT="$TMP_DIR""/qcow_mount_$RANDOM"
  local DIRS_QCOW_MOUNT=0
  FILES_QCOW_MOUNT=0
 
  if ! [[ -f "$QCOW_PATH_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
  sub_module_title "Qemu QCOW filesystem extractor"
 
  mkdir -p "$TMP_QCOW_MOUNT" 2>/dev/null || true
  print_output "[*] Trying to mount $ORANGE$QCOW_PATH_$NC to $ORANGE$TMP_QCOW_MOUNT$NC directory"
 
  # 取消nbd内核模块的挂载
  if lsmod | grep -q nbd; then
    rmmod nbd || true
  fi
 
    # 加一个锁
  if ! [[ -d /var/lock ]]; then
    mkdir /var/lock || true
  fi
 
    # 加载内核模块
  print_output "[*] Load kernel module ${ORANGE}nbd$NC."
  modprobe nbd max_part=8
  print_output "[*] Qemu disconnect device ${ORANGE}/dev/nbd$NC."
  qemu-nbd --disconnect /dev/nbd0
  print_output "[*] Qemu connect device ${ORANGE}/dev/nbd$NC."
  qemu-nbd --connect=/dev/nbd0 "$QCOW_PATH_"
 
  print_output "[*] Identification of partitions on ${ORANGE}/dev/nbd$NC."
 
    # 确保存在nbd设备
  mapfile -t NBD_DEVS < <(fdisk -l /dev/nbd0 | grep "^/dev/" | awk '{print $1}' || true)
  if [[ "${#NBD_DEVS[@]}" -eq 0 ]]; then
    # sometimes we are not able to find the partitions with fdisk -> fallback
    NBD_DEVS+=( "/dev/nbd0" )
  fi
 
  print_ln
  fdisk /dev/nbd0 -l
  print_ln
 
  for NBD_DEV in "${NBD_DEVS[@]}"; do
    print_output "[*] Extract data from partition $ORANGE$NBD_DEV$NC"
 
      # 将镜像挂载到nbd设备上
    mount "$NBD_DEV" "$TMP_QCOW_MOUNT" || true
 
    if mount | grep -q "$NBD_DEV"; then
      EXTRACTION_DIR_FINAL="$EXTRACTION_DIR_"/"$(basename "$NBD_DEV")"
 
        # 调用copy_qemu_nbd函数，下方给出分析
      copy_qemu_nbd "$TMP_QCOW_MOUNT" "$EXTRACTION_DIR_FINAL"
 
      FILES_QCOW_MOUNT=$(find "$EXTRACTION_DIR_FINAL" -type f | wc -l)
      DIRS_QCOW_MOUNT=$(find "$EXTRACTION_DIR_FINAL" -type d | wc -l)
    fi
    print_output "[*] Extracted $ORANGE$FILES_QCOW_MOUNT$NC files and $ORANGE$DIRS_QCOW_MOUNT$NC directories from the firmware image."
    write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
    write_csv_log "Qemu QCOW filesystem extractor" "$QCOW_PATH_" "$EXTRACTION_DIR_FINAL" "$FILES_QCOW_MOUNT" "$DIRS_QCOW_MOUNT" "NA"
 
    print_output "[*] Unmounting $ORANGE$TMP_QCOW_MOUNT$NC directory"
    umount "$TMP_QCOW_MOUNT"
  done
  qemu-nbd --disconnect /dev/nbd0
  rm -r "$TMP_QCOW_MOUNT"
}

copy_qemu_nbd函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
copy_qemu_nbd() {
  local SOURCE_CP="${1:-}"
  local DEST_CP="${2:-}"
  if ! [[ -d "$SOURCE_CP" ]]; then
    return
  fi
 
  print_output "[*] Copying $ORANGE$SOURCE_CP$NC to firmware tmp directory ($ORANGE$DEST_CP$NC)"
  mkdir -p "$DEST_CP" 2>/dev/null || true
 
    # 复制所有内容到提取的文件夹。
    # -p表示保留文件属性，-r表示递归复制子目录，-i表示覆盖前进行询问。
  cp -pri "$SOURCE_CP"/* "$DEST_CP" 2>/dev/null || true
  print_ln
  print_output "[*] Using the following firmware directory ($ORANGE$DEST_CP$NC) as base directory:"
 
    # 通过不跨越设备、最大深度1级子目录进行搜索，统计文件。
  find "$DEST_CP" -xdev -maxdepth 1 -ls | tee -a "$LOG_FILE"
  print_ln
}

P25_android_ota.sh

提取Android OTA更新文件
android_ota_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
android_ota_extractor() {
  local OTA_INIT_PATH_="$1"
  local EXTRACTION_DIR_="$2"
  local DIRS_OTA=0
  FILES_OTA=0
 
  if ! [[ -f "$OTA_INIT_PATH_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
  sub_module_title "Android OTA extractor"
 
  hexdump -C "$OTA_INIT_PATH_" | head | tee -a "$LOG_FILE" || true
 
  if [[ -d "$EXT_DIR"/payload_dumper ]]; then
    print_ln
    print_output "[*] Extracting Android OTA payload.bin file ..."
    print_ln
 
      # 使用扩展工具进行提取。
    python3 "$EXT_DIR"/payload_dumper/payload_dumper.py --out "$EXTRACTION_DIR_" "$OTA_INIT_PATH_" | tee -a "$LOG_FILE"
 
    FILES_OTA=$(find "$EXTRACTION_DIR_" -type f | wc -l)
    DIRS_OTA=$(find "$EXTRACTION_DIR_" -type d | wc -l)
    print_output "[*] Extracted $ORANGE$FILES_OTA$NC files and $ORANGE$DIRS_OTA$NC directories from the firmware image."
    write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
    write_csv_log "Android OTA extractor" "$OTA_INIT_PATH_" "$EXTRACTION_DIR_" "$FILES_OTA" "$DIRS_OTA" "via payload_dumper.py"
  else
    print_output "[-] Android OTA payload.bin extractor not found - check your installation"
  fi
}

P35_UEFI_extractor.sh

提取UEFI镜像与BIOSUtilities
uefi_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
uefi_extractor(){
  sub_module_title "UEFI Extractor"
 
  local FIRMWARE_PATH_="${1:-}"
  local EXTRACTION_DIR_="${2:-}"
 
  local FIRMWARE_NAME_=""
  local UEFI_EXTRACT_REPORT_FILE=""
 
  local UEFI_EXTRACT_BIN="$EXT_DIR""/UEFITool/UEFIExtract"
  local FILES_UEFI=0
  local DIRS_UEFI=0
  local NVARS=0
  local PE32_IMAGE=0
  local EFI_ARCH=""
 
  if ! [[ -f "$FIRMWARE_PATH_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
  FIRMWARE_NAME_="$(basename "$FIRMWARE_PATH_")"
  if ! [[ -d "$EXTRACTION_DIR_" ]]; then
    mkdir -p "$EXTRACTION_DIR_"
  fi
  cp "$FIRMWARE_PATH_" "$EXTRACTION_DIR_"
  "$UEFI_EXTRACT_BIN" "$EXTRACTION_DIR_"firmware all &> "$LOG_PATH_MODULE"/uefi_extractor_"$FIRMWARE_NAME_".log
  UEFI_EXTRACT_REPORT_FILE="$EXTRACTION_DIR_"firmware.report.txt
  mv "$UEFI_EXTRACT_REPORT_FILE" "$LOG_PATH_MODULE"
  UEFI_EXTRACT_REPORT_FILE="$LOG_PATH_MODULE"/firmware.report.txt
  if [[ -f "$EXTRACTION_DIR_"/firmware ]]; then
    rm "$EXTRACTION_DIR_"/firmware
  fi
 
  if [[ -f "$LOG_PATH_MODULE"/uefi_extractor_"$FIRMWARE_NAME_".log ]]; then
    tee -a "$LOG_FILE" < "$LOG_PATH_MODULE"/uefi_extractor_"$FIRMWARE_NAME_".log
  fi
 
  print_ln
  print_output "[*] Using the following firmware directory ($ORANGE${EXTRACTION_DIR_}firmware.dump$NC) as base directory:"
  find "$EXTRACTION_DIR_"firmware.dump -xdev -maxdepth 1 -ls | tee -a "$LOG_FILE"
  print_ln
 
  NVARS=$(grep -c "NVAR entry" "$UEFI_EXTRACT_REPORT_FILE" || true)
  PE32_IMAGE=$(grep -c "PE32 image" "$UEFI_EXTRACT_REPORT_FILE" || true)
  DRIVER_COUNT=$(grep -c "DXE driver" "$UEFI_EXTRACT_REPORT_FILE" || true)
  EFI_ARCH=$(find "$EXTRACTION_DIR_" -name 'info.txt' -exec grep 'Machine type:' {} \; | sed -E 's/Machine\ type\:\ //g' | uniq | head -n 1)
 
  if [[ -n "$EFI_ARCH" ]]; then
    print_output "[*] Found $ORANGE$PE32_IMAGE$NC PE32 images for architecture $ORANGE$EFI_ARCH$NC drivers."
    print_output "[+] Possible architecture details found ($ORANGE UEFI Extractor $GREEN): $ORANGE$EFI_ARCH$NC"
    export EFI_ARCH
    backup_var "EFI_ARCH" "$EFI_ARCH"
  fi
 
  FILES_UEFI=$(grep -c "File" "$UEFI_EXTRACT_REPORT_FILE" || true)
  DIRS_UEFI=$(find "$EXTRACTION_DIR_" -type d | wc -l)
  print_output "[*] Extracted $ORANGE$FILES_UEFI$NC files and $ORANGE$DIRS_UEFI$NC directories from the firmware image."
  print_output "[*] Found $ORANGE$NVARS$NC NVARS and $ORANGE$DRIVER_COUNT$NC drivers."
  write_csv_log "Extractor module" "Original file" "extracted file/dir" "file counter" "directory counter" "further details"
  write_csv_log "UEFI extractor" "$FIRMWARE_PATH_" "$EXTRACTION_DIR_" "$FILES_UEFI" "$DIRS_UEFI" "NA"
  print_ln
}

P59_binwalk_extractor.sh

分析固件与binwalk，检查熵和提取固件到日志目录
P59_binwalk_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
P59_binwalk_extractor() {
  module_log_init "${FUNCNAME[0]}"
  module_title "Binwalk firmware extractor"
  pre_module_reporter "${FUNCNAME[0]}"
 
  export LINUX_PATH_COUNTER=0
 
    # 重新定义unblob下的sasquatch工具指向binwalk
  # we need to check if sasquatch is the correct one for binwalk:
  if ! [[ "$(readlink -q -f "$UNBLOB_PATH"/sasquatch)" == "/usr/local/bin/sasquatch_binwalk" ]]; then
    if [[ -L "$UNBLOB_PATH"/sasquatch ]]; then
      rm "$UNBLOB_PATH"/sasquatch
    fi
    ln -s /usr/local/bin/sasquatch_binwalk "$UNBLOB_PATH"/sasquatch || true
  fi
 
  # typically FIRMWARE_PATH is only a file if none of the EMBA extractors were able to extract something
  # This means we are using binwalk in Matryoshka mode here
  # if we have a directory with multiple files in it we automatically pass here and run into the deep extractor
  if [[ -f "$FIRMWARE_PATH" ]]; then
 
      # 首先调用binwalking，此函数将在下方给出分析。
        # 作用：对固件进行简单地分析并给出熵图。
    # we love binwalk ... this is our first chance for extracting everything
    binwalking "$FIRMWARE_PATH"
  fi
 
    # 将在下方给出分析
  # 功能：通过特殊二进制路径来推测文件系统根目录在哪
  # FIRMWARE_PATH_CP is typically /log/firmware - shellcheck is probably confused here
  # shellcheck disable=SC2153
  detect_root_dir_helper "$FIRMWARE_PATH_CP"
 
  print_ln
 
  # 统计提取的文件
  FILES_EXT=$(find "$FIRMWARE_PATH_CP" -xdev -type f | wc -l )
  UNIQUE_FILES=$(find "$FIRMWARE_PATH_CP" "${EXCL_FIND[@]}" -xdev -type f -exec md5sum {} \; 2>/dev/null | sort -u -k1,1 | cut -d\  -f3 | wc -l )
  DIRS_EXT=$(find "$FIRMWARE_PATH_CP" -xdev -type d | wc -l )
  BINS=$(find "$FIRMWARE_PATH_CP" "${EXCL_FIND[@]}" -xdev -type f -exec file {} \; | grep -c "ELF" || true)
 
  if [[ "$BINS" -gt 0 || "$UNIQUE_FILES" -gt 0 ]]; then
    sub_module_title "Firmware extraction details"
 
      # 将在下方给出分析
      # 主要功能：用于统计linux文件系统中基本的特征文件或文件名
    linux_basic_identification_helper "$FIRMWARE_PATH_CP"
    print_ln
    print_output "[*] Found $ORANGE$FILES_EXT$NC files ($ORANGE$UNIQUE_FILES$NC unique files) and $ORANGE$DIRS_EXT$NC directories at all."
    print_output "[*] Found $ORANGE$BINS$NC binaries."
    print_output "[*] Additionally the Linux path counter is $ORANGE$LINUX_PATH_COUNTER$NC."
    print_ln
    tree -csh "$FIRMWARE_PATH_CP" | tee -a "$LOG_FILE"
 
    # now it should be fine to also set the FIRMWARE_PATH ot the FIRMWARE_PATH_CP
    export FIRMWARE_PATH="$FIRMWARE_PATH_CP"
 
    if [[ "${#ROOT_PATH[@]}" -gt 0 ]] ; then
      write_csv_log "FILES" "UNIQUE_FILES" "DIRS" "Binaries" "LINUX_PATH_COUNTER" "Root PATH detected"
      for R_PATH in "${ROOT_PATH[@]}"; do
        write_csv_log "$FILES_EXT" "$UNIQUE_FILES" "$DIRS_EXT" "$BINS" "$LINUX_PATH_COUNTER" "$R_PATH"
      done
    fi
    backup_var "FILES_EXT" "$FILES_EXT"
    backup_var "FILES_EXT" "$UNIQUE_FILES"
    backup_var "FILES_EXT" "$DIRS_EXT"
  fi
 
  module_end_log "${FUNCNAME[0]}" "$FILES_EXT"
}

binwalking函数：
对固件进行简单地分析并给出熵图。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# this function is for the first round of binwalk
# in case no other EMBA extractor did something and the
# current firmware file is a file and not multiple files
binwalking() {
  local FIRMWARE_PATH_="${1:-}"
  export OUTPUT_DIR_BINWALK=""
 
  if ! [[ -f "$FIRMWARE_PATH_" ]]; then
    print_output "[-] No file for extraction provided"
    return
  fi
 
  sub_module_title "Analyze binary firmware blob with binwalk"
 
    # 简单地用binwalk分析一下
  print_output "[*] Basic analysis with binwalk"
  binwalk "$FIRMWARE_PATH_" | tee -a "$LOG_FILE"
 
  print_ln "no_log"
  # we use the original FIRMWARE_PATH for entropy testing, just if it is a file
  if [[ -f $FIRMWARE_PATH_BAK ]] && ! [[ -f "$LOG_DIR"/firmware_entropy.png ]]; then
    print_output "[*] Entropy testing with binwalk ... "
    # we have to change the working directory for binwalk, because everything except the log directory is read-only in
    # Docker container and binwalk fails to save the entropy picture there
    if [[ $IN_DOCKER -eq 1 ]] ; then
      cd "$LOG_DIR" || return
 
        # 用binwalk生成熵图
      print_output "$(binwalk -E -F -J "$FIRMWARE_PATH_BAK")"
      mv "$(basename "$FIRMWARE_PATH_".png)" "$LOG_DIR"/firmware_entropy.png 2> /dev/null || true
      cd /emba || return
    else
      print_output "$(binwalk -E -F -J "$FIRMWARE_PATH_BAK")"
      mv "$(basename "$FIRMWARE_PATH_".png)" "$LOG_DIR"/firmware_entropy.png 2> /dev/null || true
    fi
  fi
 
  OUTPUT_DIR_BINWALK=$(basename "$FIRMWARE_PATH_")
  OUTPUT_DIR_BINWALK="$FIRMWARE_PATH_CP""/""$OUTPUT_DIR_BINWALK"_binwalk_emba
 
  print_ln
  print_output "[*] Extracting firmware to directory $ORANGE$OUTPUT_DIR_BINWALK$NC"
  # this is not working in background. I have created a new function that gets executed in the background
  # probably there is a more elegant way
  # binwalk is executed in Matryoshka mode
  binwalk_deep_extract_helper 1 "$FIRMWARE_PATH_" "$OUTPUT_DIR_BINWALK" &
  WAIT_PIDS+=( "$!" )
  wait_for_extractor
  WAIT_PIDS=( )
 
  MD5_DONE_DEEP+=( "$(md5sum "$FIRMWARE_PATH_" | awk '{print $1}')" )
}

detect_root_dir_helper函数：
通过特殊二进制路径来推测文件系统根目录在哪

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
detect_root_dir_helper() {
  SEARCH_PATH="${1:-}"
 
  print_output "[*] Root directory auto detection for $ORANGE$SEARCH_PATH$NC (could take some time)\\n"
  export ROOT_PATH=()
  local R_PATH
  local MECHANISM=""
 
  # 过滤出解释器的可执行文件
  mapfile -t INTERPRETER_FULL_PATH < <(find "$SEARCH_PATH" -ignore_readdir_race -type f -exec file {} \; 2>/dev/null | grep "ELF" | grep "interpreter" | sed s/.*interpreter\ // | sed s/,\ .*$// | sort -u 2>/dev/null || true)
 
    # 获取解释器路径
  if [[ "${#INTERPRETER_FULL_PATH[@]}" -gt 0 ]]; then
    for INTERPRETER_PATH in "${INTERPRETER_FULL_PATH[@]}"; do
      # now we have a result like this "/lib/ld-uClibc.so.0"
      # lets escape it
        # 将/替换为\/
      INTERPRETER_ESCAPED=$(echo "$INTERPRETER_PATH" | sed -e 's/\//\\\//g')
      mapfile -t INTERPRETER_FULL_RPATH < <(find "$SEARCH_PATH" -ignore_readdir_race -wholename "*$INTERPRETER_PATH" 2>/dev/null | sort -u)
      for R_PATH in "${INTERPRETER_FULL_RPATH[@]}"; do
        # remove the interpreter path from the full path:
        R_PATH="${R_PATH//$INTERPRETER_ESCAPED/}"
        if [[ -v R_PATH ]] && [[ -d "$R_PATH" ]]; then
          ROOT_PATH+=( "$R_PATH" )
          MECHANISM="binary interpreter"
        fi
      done
    done
  fi
 
    # 找到所有解释器的根目录
  # if we can't find the interpreter we fall back to a search for something like "*root/bin/* and take this:
  mapfile -t ROOTx_PATH < <(find "$SEARCH_PATH" -xdev \( -path "*extracted/bin" -o -path "*root/bin" \) -exec dirname {} \; 2>/dev/null)
  for R_PATH in "${ROOTx_PATH[@]}"; do
    if [[ -d "$R_PATH" ]]; then
      ROOT_PATH+=( "$R_PATH" )
      if [[ -z "$MECHANISM" ]]; then
        MECHANISM="dir names"
      elif [[ -n "$MECHANISM" ]] && ! echo "$MECHANISM" | grep -q "dir names"; then
        MECHANISM="$MECHANISM / dir names"
      fi
    fi
  done
 
  mapfile -t ROOTx_PATH < <(find "$SEARCH_PATH" -xdev \( -path "*/sbin" -o -path "*/bin" -o -path "*/lib" -o -path "*/etc" -o -path "*/root" -o -path "*/dev" -o -path "*/opt" -o -path "*/proc" -o -path "*/lib64" -o -path "*/boot" -o -path "*/home" \) -exec dirname {} \; | sort | uniq -c | sort -r)
  for R_PATH in "${ROOTx_PATH[@]}"; do
    CNT=$(echo "$R_PATH" | awk '{print $1}')
    if [[ "$CNT" -lt 5 ]]; then
      # we only use paths with more then 4 matches as possible root path
      continue
    fi
    R_PATH=$(echo "$R_PATH" | awk '{print $2}')
    if [[ -d "$R_PATH" ]]; then
      ROOT_PATH+=( "$R_PATH" )
      if [[ -z "$MECHANISM" ]]; then
        MECHANISM="dir names"
      elif [[ -n "$MECHANISM" ]] && ! echo "$MECHANISM" | grep -q "dir names"; then
        MECHANISM="$MECHANISM / dir names"
      fi
    fi
  done
 
  mapfile -t ROOTx_PATH < <(find "$SEARCH_PATH" -xdev -path "*bin/busybox" | sed -E 's/\/.?bin\/busybox//')
  for R_PATH in "${ROOTx_PATH[@]}"; do
    if [[ -d "$R_PATH" ]]; then
      ROOT_PATH+=( "$R_PATH" )
      if [[ -z "$MECHANISM" ]]; then
        MECHANISM="busybox"
      elif [[ -n "$MECHANISM" ]] && ! echo "$MECHANISM" | grep -q "busybox"; then
        MECHANISM="$MECHANISM / busybox"
      fi
    fi
  done
 
  mapfile -t ROOTx_PATH < <(find "$SEARCH_PATH" -xdev -path "*bin/bash" -exec file {} \; | grep "ELF" | cut -d: -f1 | sed -E 's/\/.?bin\/bash//' || true)
  for R_PATH in "${ROOTx_PATH[@]}"; do
    if [[ -d "$R_PATH" ]]; then
      ROOT_PATH+=( "$R_PATH" )
      if [[ -z "$MECHANISM" ]]; then
        MECHANISM="shell"
      elif [[ -n "$MECHANISM" ]] && ! echo "$MECHANISM" | grep -q "shell"; then
        MECHANISM="$MECHANISM / shell"
      fi
    fi
  done
 
  mapfile -t ROOTx_PATH < <(find "$SEARCH_PATH" -xdev -path "*bin/sh" -exec file {} \; | grep "ELF" | cut -d: -f1 | sed -E 's/\/.?bin\/sh//' || true)
  for R_PATH in "${ROOTx_PATH[@]}"; do
    if [[ -d "$R_PATH" ]]; then
      ROOT_PATH+=( "$R_PATH" )
      if [[ -z "$MECHANISM" ]]; then
        MECHANISM="shell"
      elif [[ -n "$MECHANISM" ]] && ! echo "$MECHANISM" | grep -q "shell"; then
        MECHANISM="$MECHANISM / shell"
      fi
    fi
  done
 
  if [[ ${#ROOT_PATH[@]} -eq 0 ]]; then
    export RTOS=1
    ROOT_PATH+=( "$SEARCH_PATH" )
    MECHANISM="last resort"
  else
    export RTOS=0
  fi
 
  eval "ROOT_PATH=($(for i in "${ROOT_PATH[@]}" ; do echo "\"$i\"" ; done | sort -u))"
  if [[ -v ROOT_PATH[@] && "$RTOS" -eq 0 ]]; then
    print_output "[*] Found $ORANGE${#ROOT_PATH[@]}$NC different root directories:"
    write_link "s05#file_dirs"
  fi
 
  for R_PATH in "${ROOT_PATH[@]}"; do
    if [[ "$MECHANISM" == "last resort" ]]; then
      print_output "[*] Found no real root directory - setting it to: $ORANGE$R_PATH$NC via $ORANGE$MECHANISM$NC."
    else
      print_output "[+] Found the following root directory: $ORANGE$R_PATH$GREEN via $ORANGE$MECHANISM$GREEN."
    fi
    write_link "s05#file_dirs"
  done
}

linux_basic_identification_helper函数：
用于统计linux文件系统中基本的特征文件或文件名

1
2
3
4
5
6
7
8
9
linux_basic_identification_helper() {
  local FIRMWARE_PATH_CHECK="${1:-}"
  if ! [[ -d "$FIRMWARE_PATH_CHECK" ]]; then
    LINUX_PATH_COUNTER=0
    return
  fi
  LINUX_PATH_COUNTER="$(find "$FIRMWARE_PATH_CHECK" "${EXCL_FIND[@]}" -xdev -type d -iname bin -o -type f -iname busybox -o -type f -name shadow -o -type f -name passwd -o -type d -iname sbin -o -type d -iname etc 2> /dev/null | wc -l)"
  backup_var "LINUX_PATH_COUNTER" "$LINUX_PATH_COUNTER"
}

P60_firmware_bin_extractor.sh

分析固件与binwalk，检查熵和提取固件到日志目录
P60_firmware_bin_extractor函数：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
P60_firmware_bin_extractor() {
  module_log_init "${FUNCNAME[0]}"
  module_title "Binary firmware deep extractor"
  pre_module_reporter "${FUNCNAME[0]}"
 
  export DISK_SPACE_CRIT=0
 
  # If we have not found a linux filesystem we try to do an extraction round on every file multiple times
  if [[ $RTOS -eq 0 ]] ; then
    module_end_log "${FUNCNAME[0]}" 0
    return
  fi
 
  # 跟前面一样，检查sasquatch链接
  # we need to check if sasquatch is the correct one for binwalk:
  if ! [[ "$(readlink -q -f "$UNBLOB_PATH"/sasquatch)" == "/usr/local/bin/sasquatch_binwalk" ]]; then
    if [[ -L "$UNBLOB_PATH"/sasquatch ]]; then
      rm "$UNBLOB_PATH"/sasquatch
    fi
    ln -s /usr/local/bin/sasquatch_binwalk "$UNBLOB_PATH"/sasquatch || true
  fi
 
    # 用于检查磁盘空间，如果没超过最大的额外空间，就会开启深度提取
  check_disk_space
  if ! [[ "$DISK_SPACE" -gt "$MAX_EXT_SPACE" ]]; then
    deep_extractor
  else
    print_output "[!] $(date) - Extractor needs too much disk space $DISK_SPACE" "main"
    print_output "[!] $(date) - Ending extraction processes - no deep extraction performed" "main"
    DISK_SPACE_CRIT=1
  fi
 
  print_ln
 
  FILES_EXT=$(find "$FIRMWARE_PATH_CP" -xdev -type f | wc -l )
  UNIQUE_FILES=$(find "$FIRMWARE_PATH_CP" "${EXCL_FIND[@]}" -xdev -type f -exec md5sum {} \; 2>/dev/null | sort -u -k1,1 | cut -d\  -f3 | wc -l )
  DIRS_EXT=$(find "$FIRMWARE_PATH_CP" -xdev -type d | wc -l )
  BINS=$(find "$FIRMWARE_PATH_CP" "${EXCL_FIND[@]}" -xdev -type f -exec file {} \; | grep -c "ELF" || true)
 
  if [[ "$BINS" -gt 0 || "$UNIQUE_FILES" -gt 0 ]]; then
    export LINUX_PATH_COUNTER=0
    linux_basic_identification_helper "$FIRMWARE_PATH_CP"
    print_ln
    print_output "[*] Found $ORANGE$FILES_EXT$NC files ($ORANGE$UNIQUE_FILES$NC unique files) and $ORANGE$DIRS_EXT$NC directories at all."
    print_output "[*] Found $ORANGE$BINS$NC binaries."
    print_output "[*] Additionally the Linux path counter is $ORANGE$LINUX_PATH_COUNTER$NC."
    print_ln
    tree -csh "$FIRMWARE_PATH_CP" | tee -a "$LOG_FILE"
 
    # now it should be fine to also set the FIRMWARE_PATH ot the FIRMWARE_PATH_CP
    export FIRMWARE_PATH="$FIRMWARE_PATH_CP"
 
    if [[ "${#ROOT_PATH[@]}" -gt 0 ]] ; then
      write_csv_log "FILES" "UNIQUE_FILES" "DIRS" "Binaries" "LINUX_PATH_COUNTER" "Root PATH detected"
      for R_PATH in "${ROOT_PATH[@]}"; do
        write_csv_log "$FILES_EXT" "$UNIQUE_FILES" "$DIRS_EXT" "$BINS" "$LINUX_PATH_COUNTER" "$R_PATH"
      done
    fi
    backup_var "FILES_EXT" "$FILES_EXT"
  fi
 
  module_end_log "${FUNCNAME[0]}" "$FILES_EXT"
}

check_disk_space函数：
用于检查磁盘空间

1
2
3
4
check_disk_space() {
  export DISK_SPACE
  DISK_SPACE=$(du -hm "$FIRMWARE_PATH_CP" --max-depth=1 --exclude="proc" 2>/dev/null | awk '{ print $1 }' | sort -hr | head -1 || true)
}

deep_extractor函数：
深度提取模式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
deep_extractor() {
  sub_module_title "Deep extraction mode"
 
  FILE_MD5=""
 
  FILES_BEFORE_DEEP=$(find "$FIRMWARE_PATH_CP" -xdev -type f | wc -l )
 
  # if we run into the deep extraction mode we always do at least one extraction round:
  if [[ "$DISK_SPACE_CRIT" -eq 0 ]]; then
    print_output "[*] Deep extraction - 1st round"
    print_output "[*] Walking through all files and try to extract what ever possible"
 
      # 进行深度提取，将在下方给出分析
    deeper_extractor_helper
    detect_root_dir_helper "$FIRMWARE_PATH_CP"
  fi
 
    # 进行多轮提取
  if [[ $RTOS -eq 1 && "$DISK_SPACE_CRIT" -eq 0 ]]; then
    print_output "[*] Deep extraction - 2nd round"
    print_output "[*] Walking through all files and try to extract what ever possible"
 
    deeper_extractor_helper
    detect_root_dir_helper "$FIRMWARE_PATH_CP"
  fi
 
  if [[ $RTOS -eq 1 && "$DISK_SPACE_CRIT" -eq 0 ]]; then
    print_output "[*] Deep extraction - 3rd round"
    print_output "[*] Walking through all files and try to extract what ever possible"
 
    deeper_extractor_helper
    detect_root_dir_helper "$FIRMWARE_PATH_CP"
  fi
 
  if [[ $RTOS -eq 1 && "$DISK_SPACE_CRIT" -eq 0 ]]; then
    print_output "[*] Deep extraction - 4th round"
    print_output "[*] Walking through all files and try to extract what ever possible with binwalk matryoshka mode"
    print_output "[*] WARNING: This is the last extraction round that is executed."
 
    # if we are already that far we do a final matryoshka extraction mode
    deeper_extractor_helper "M"
    detect_root_dir_helper "$FIRMWARE_PATH_CP"
  fi
 
  FILES_AFTER_DEEP=$(find "$FIRMWARE_PATH_CP" -xdev -type f | wc -l )
 
  print_output "[*] Before deep extraction we had $ORANGE$FILES_BEFORE_DEEP$NC files, after deep extraction we have now $ORANGE$FILES_AFTER_DEEP$NC files extracted."
}

deeper_extractor_helper函数：
进行深度提取

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
deeper_extractor_helper() {
 
  if [[ -v 1 ]] && [[ "$1" == "M" ]]; then
    local MATRYOSHKA=1
  else
    local MATRYOSHKA=0
  fi
  local FILE_TMP=""
  local FILE_MD5=""
 
    # 将特定文件排除，并对文件本身进行去重，记录路径到FILE_ARR_LIMITED中
  prepare_file_arr_limited "$FIRMWARE_PATH_CP"
 
  # 遍历每一个独立文件
  for FILE_TMP in "${FILE_ARR_LIMITED[@]}"; do
 
    FILE_MD5="$(md5sum "$FILE_TMP" | awk '{print $1}')"
    # let's check the current md5sum against our array of unique md5sums - if we have a match this is already extracted
    # already extracted stuff is now ignored
 
    if [[ ! " ${MD5_DONE_DEEP[*]} " =~ ${FILE_MD5} ]]; then
 
      print_output "[*] Details of file: $ORANGE$FILE_TMP$NC"
      print_output "$(indent "$(file "$FILE_TMP")")"
 
      # do a quick check if EMBA should handle the file or we give it to binwalk:
      # fw_bin_detector is a function from p02
      fw_bin_detector "$FILE_TMP"
 
        # 进行一轮提取
      if [[ "$VMDK_DETECTED" -eq 1 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          vmdk_extractor "$FILE_TMP" "${FILE_TMP}_vmdk_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          vmdk_extractor "$FILE_TMP" "${FILE_TMP}_vmdk_extracted"
        fi
      elif [[ "$UBI_IMAGE" -eq 1 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          ubi_extractor "$FILE_TMP" "${FILE_TMP}_ubi_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          ubi_extractor "$FILE_TMP" "${FILE_TMP}_ubi_extracted"
        fi
      elif [[ "$DLINK_ENC_DETECTED" -eq 1 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          dlink_SHRS_enc_extractor "$FILE_TMP" "${FILE_TMP}_shrs_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          dlink_SHRS_enc_extractor "$FILE_TMP" "${FILE_TMP}_shrs_extracted"
        fi
      elif [[ "$DLINK_ENC_DETECTED" -eq 2 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          dlink_enc_img_extractor "$FILE_TMP" "${FILE_TMP}_enc_img_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          dlink_enc_img_extractor "$FILE_TMP" "${FILE_TMP}_enc_img_extracted"
        fi
      elif [[ "$EXT_IMAGE" -eq 1 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          ext_extractor "$FILE_TMP" "${FILE_TMP}_ext_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          ext_extractor "$FILE_TMP" "${FILE_TMP}_ext_extracted"
        fi
      elif [[ "$ENGENIUS_ENC_DETECTED" -ne 0 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          engenius_enc_extractor "$FILE_TMP" "${FILE_TMP}_engenius_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          engenius_enc_extractor "$FILE_TMP" "${FILE_TMP}_engenius_extracted"
        fi
      elif [[ "$BSD_UFS" -ne 0 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          ufs_extractor "$FILE_TMP" "${FILE_TMP}_bsd_ufs_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          ufs_extractor "$FILE_TMP" "${FILE_TMP}_bsd_ufs_extracted"
        fi
      elif [[ "$ANDROID_OTA" -ne 0 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          android_ota_extractor "$FILE_TMP" "${FILE_TMP}_android_ota_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          android_ota_extractor "$FILE_TMP" "${FILE_TMP}_android_ota_extracted"
        fi
      elif [[ "$OPENSSL_ENC_DETECTED" -ne 0 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          foscam_enc_extractor "$FILE_TMP" "${FILE_TMP}_foscam_enc_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          foscam_enc_extractor "$FILE_TMP" "${FILE_TMP}_foscam_enc_extracted"
        fi
      elif [[ "$BUFFALO_ENC_DETECTED" -ne 0 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          buffalo_enc_extractor "$FILE_TMP" "${FILE_TMP}_buffalo_enc_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          buffalo_enc_extractor "$FILE_TMP" "${FILE_TMP}_buffalo_enc_extracted"
        fi
      elif [[ "$ZYXEL_ZIP" -ne 0 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          zyxel_zip_extractor "$FILE_TMP" "${FILE_TMP}_zyxel_enc_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          zyxel_zip_extractor "$FILE_TMP" "${FILE_TMP}_zyxel_enc_extracted"
        fi
      elif [[ "$QCOW_DETECTED" -ne 0 ]]; then
        if [[ "$THREADED" -eq 1 ]]; then
          qcow_extractor "$FILE_TMP" "${FILE_TMP}_qemu_qcow_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          qcow_extractor "$FILE_TMP" "${FILE_TMP}_qemu_qcow_extracted"
        fi
 
      else
        # default case to binwalk
        if [[ "$THREADED" -eq 1 ]]; then
          binwalk_deep_extract_helper "$MATRYOSHKA" "$FILE_TMP" "${FILE_TMP}_binwalk_extracted" &
          BIN_PID="$!"
          store_kill_pids "$BIN_PID"
          disown "$BIN_PID" 2> /dev/null || true
          WAIT_PIDS_P20+=( "$BIN_PID" )
        else
          binwalk_deep_extract_helper "$MATRYOSHKA" "$FILE_TMP" "${FILE_TMP}_binwalk_extracted"
        fi
      fi
 
      MD5_DONE_DEEP+=( "$FILE_MD5" )
      max_pids_protection "$MAX_MOD_THREADS" "${WAIT_PIDS_P20[@]}"
    fi
 
    check_disk_space
 
    FREE_SPACE=$(df --output=avail "$LOG_DIR" | awk 'NR==2')
    if [[ "$FREE_SPACE" -lt 100000 ]]; then
      # this stops the complete EMBA test
      print_output "[!] $(date) - The system is running out of disk space $ORANGE$FREE_SPACE$NC" "main"
      print_output "[!] $(date) - Ending EMBA firmware analysis processes" "main"
      cleaner 1
      exit
    elif [[ "$DISK_SPACE" -gt "$MAX_EXT_SPACE" ]]; then
      # this stops the deep extractor but not EMBA
      print_output "[!] $(date) - Extractor needs too much disk space $DISK_SPACE" "main"
      print_output "[!] $(date) - Ending extraction processes" "main"
      DISK_SPACE_CRIT=1
      break
    fi
  done
 
  [[ "$THREADED" -eq 1 ]] && wait_for_pid "${WAIT_PIDS_P20[@]}"
}

P65_package_extractor.sh

识别和提取典型的软件包档案，如deb, apk, ipk
逻辑比较简单，识别文件名然后针对性解包

P70_unblob.sh

将带有unblob的固件提取到模块日志目录（仅用于评估）
逻辑比较简单，直接使用unblob进行提取，然后识别文件系统根目录、统计信息。

P99_prepare_analyzer.sh

一些准备工作（检查固件、架构检查等）
检查固件、准备二进制文件列表、架构检查、文件系统根目录识别、设置配置文件路径、准备文件列表。


[培训]二进制漏洞攻防（第3期）；满10人开班；模糊测试与工具使用二次开发；网络协议漏洞挖掘；Linux内核漏洞挖掘与利用；AOSP漏洞挖掘与利用；代码审计。

收藏
点赞・3
打赏
分享
最新回复 (1)
秋狝
雪    币： 19299
活跃值： (28933)
能力值： ( LV2，RANK：10 )
在线值：
发帖
回帖
粉丝
私信
秋狝 2023-5-2 17:14
2 楼
1
感谢分享
游客
登录 | 注册 方可回帖
　回帖　 表情 雪币赚取及消费
高级回复
返回