首页
社区
课程
招聘
[原创]菜鸟学8.1版本dex加载流程笔记--第一篇:oat_file_manager与oat_file_assistant
发表于: 2020-3-1 09:13 8979

[原创]菜鸟学8.1版本dex加载流程笔记--第一篇:oat_file_manager与oat_file_assistant

2020-3-1 09:13
8979

菜鸟最近初学脱壳,必须得先搞明白dex的加载流程。n6装的8.1,最近跟了一遍8.1源码,记录笔记,理解一下类似opencommon的脱壳点

网上的文章大都比较老了,参考这篇6.0的加载https://www.jianshu.com/p/20dcfcf27004

新版本多走了 oat_file_manager.cc ,oat_file_assistant.cc这2个重要的类,

而没有去走linker,需要重新跟一次,在调试一遍才能搞明白整个流程。java层肯定走classloader,最终调用了dalvik_system_DexFile.cc的opeDexFileNative,进入so层,从这里开始分析

先放个简要流程,源码分别在dalvik_system_DexFile.cc ,oat_file_manager.cc ,oat_file_assistant.cc,oat_file.cc,dex_file.cc里,大致就是先得到Oat,通过Oat得到 OatDexFile,在通过 OatDexFile得到 DexFile

下面开始一步一步分析

1.首先是dalvik_system_DexFile.cc的opeDexFileNative

关键是这一句

dex_files = runtime->GetOatFileManager().OpenDexFilesFromOat(sourceName.c_str(),class_loader,dex_elements,/*out*/ &oat_file,/*out*/ &error_msgs);

与之前如6.0不同, OpenDexFilesFromOat 这个函数放在了 oat_file_manager.cc 而不是class_linker.cc

2.进入oat_file_manager.cc的 OpenDexFilesFromOat

2.1,先通过:std::unique_ptr<const OatFile> oat_file(oat_file_assistant.GetBestOatFile().release())这句获得了oat_file,

2.2然后通过:dex_files = oat_file_assistant.LoadDexFiles(*source_oat_file, dex_location)这里通过加载source_oat_file获得dex_files

2.3如果上面2.1与2.2不成立:DexFile::Open(dex_location, dex_location, kVerifyChecksum, /*out*/ &error_msg, &dex_files)) //如果LoadDexFiles上面没有获得dex_files,直接DexFile::Open打开加载原始的dexfile


opeDexFileNative    dalvik_system_DexFile.cc 
{
  OpenDexFilesFromOat    oat_file_manager.cc 
  {
   GetBestOatFile     oat_file_assistant.cc
    {
      OatFile::Open    oat_file.cc
    }
   LoadDexFiles     oat_file_assistant.cc
    {
      GetOatDexFile(得到上面open的OatFile的OatDexFile)oat_file.cc
	OpenDexFile                   oat_file.cc
       {
	  DexFile::Open           dex_file.cc
       } 
    }
  }
}

下面开始一步一步分析

1.首先是dalvik_system_DexFile.cc的opeDexFileNative

关键是这一句

dex_files = runtime->GetOatFileManager().OpenDexFilesFromOat(sourceName.c_str(),class_loader,dex_elements,/*out*/ &oat_file,/*out*/ &error_msgs);

与之前如6.0不同, OpenDexFilesFromOat 这个函数放在了 oat_file_manager.cc 而不是class_linker.cc

static jobject DexFile_openDexFileNative(JNIEnv* env,
                                         jclass,
                                         jstring javaSourceName,
                                         jstring javaOutputName ATTRIBUTE_UNUSED,
                                         jint flags ATTRIBUTE_UNUSED,
                                         jobject class_loader,
                                         jobjectArray dex_elements) {
  ScopedUtfChars sourceName(env, javaSourceName);
  if (sourceName.c_str() == nullptr) {
    return 0;
  }

  Runtime* const runtime = Runtime::Current();
  ClassLinker* linker = runtime->GetClassLinker(); //获得linker
  std::vector<std::unique_ptr<const DexFile>> dex_files;
  std::vector<std::string> error_msgs;
  const OatFile* oat_file = nullptr;

  dex_files = runtime->GetOatFileManager().OpenDexFilesFromOat(sourceName.c_str(),
                                                               class_loader,
                                                               dex_elements,
                                                               /*out*/ &oat_file,
                                                               /*out*/ &error_msgs);//这句调用OpenDexFilesFromOat获得dex_files

  if (!dex_files.empty()) {
    jlongArray array = ConvertDexFilesToJavaArray(env, oat_file, dex_files);//这句把DexFiles转化为JavaArray,方便java层使用
    if (array == nullptr) {
      ScopedObjectAccess soa(env);
      for (auto& dex_file : dex_files) {
        if (linker->IsDexFileRegistered(soa.Self(), *dex_file)) {  //这里linker只用来判断dex_file是不是已经存在,跟以前版本不同,需要注意一下
          dex_file.release();
        }
      }
    }
    return array;
  } else {
    ScopedObjectAccess soa(env);
    CHECK(!error_msgs.empty());
    // The most important message is at the end. So set up nesting by going forward, which will
    // wrap the existing exception as a cause for the following one.
    auto it = error_msgs.begin();
    auto itEnd = error_msgs.end();
    for ( ; it != itEnd; ++it) {
      ThrowWrappedIOException("%s", it->c_str());
    }

    return nullptr;
  }
}

2.进入oat_file_manager.cc的 OpenDexFilesFromOat

2.1,先通过:std::unique_ptr<const OatFile> oat_file(oat_file_assistant.GetBestOatFile().release())这句获得了oat_file,

2.2然后通过:dex_files = oat_file_assistant.LoadDexFiles(*source_oat_file, dex_location)这里通过加载source_oat_file获得dex_files

2.3如果上面2.1与2.2不成立:DexFile::Open(dex_location, dex_location, kVerifyChecksum, /*out*/ &error_msg, &dex_files)) //如果LoadDexFiles上面没有获得dex_files,直接DexFile::Open打开加载原始的dexfile

std::vector<std::unique_ptr<const DexFile>> OatFileManager::OpenDexFilesFromOat(
    const char* dex_location,
    jobject class_loader,
    jobjectArray dex_elements,
    const OatFile** out_oat_file,
    std::vector<std::string>* error_msgs) {
  ScopedTrace trace(__FUNCTION__);
  CHECK(dex_location != nullptr);
  CHECK(error_msgs != nullptr);

  // Verify we aren't holding the mutator lock, which could starve GC if we
  // have to generate or relocate an oat file.
  Thread* const self = Thread::Current();
  Locks::mutator_lock_->AssertNotHeld(self);
  Runtime* const runtime = Runtime::Current();

  OatFileAssistant oat_file_assistant(dex_location,
                                      kRuntimeISA,
                                      !runtime->IsAotCompiler());

  // Lock the target oat location to avoid races generating and loading the
  // oat file.
  std::string error_msg;
  if (!oat_file_assistant.Lock(/*out*/&error_msg)) {
    // Don't worry too much if this fails. If it does fail, it's unlikely we
    // can generate an oat file anyway.
    VLOG(class_linker) << "OatFileAssistant::Lock: " << error_msg;
  }

  const OatFile* source_oat_file = nullptr;

  if (!oat_file_assistant.IsUpToDate()) {
    // Update the oat file on disk if we can, based on the --compiler-filter
    // option derived from the current runtime options.
    // This may fail, but that's okay. Best effort is all that matters here.
    switch (oat_file_assistant.MakeUpToDate(/*profile_changed*/false, /*out*/ &error_msg)) {
      case OatFileAssistant::kUpdateFailed:
        LOG(WARNING) << error_msg;
        break;

      case OatFileAssistant::kUpdateNotAttempted:
        // Avoid spamming the logs if we decided not to attempt making the oat
        // file up to date.
        VLOG(oat) << error_msg;
        break;

      case OatFileAssistant::kUpdateSucceeded:
        // Nothing to do.
        break;
    }
  }

  // Get the oat file on disk.
  std::unique_ptr<const OatFile> oat_file(oat_file_assistant.GetBestOatFile().release());//这句获得了oat_file,下面LoadDexFiles使用这个oat_file获得dex_files

  if (oat_file != nullptr) {
    // Take the file only if it has no collisions, or we must take it because of preopting.
    bool accept_oat_file =
        !HasCollisions(oat_file.get(), class_loader, dex_elements, /*out*/ &error_msg);
    if (!accept_oat_file) {
      // Failed the collision check. Print warning.
      if (Runtime::Current()->IsDexFileFallbackEnabled()) {
        if (!oat_file_assistant.HasOriginalDexFiles()) {
          // We need to fallback but don't have original dex files. We have to
          // fallback to opening the existing oat file. This is potentially
          // unsafe so we warn about it.
          accept_oat_file = true;

          LOG(WARNING) << "Dex location " << dex_location << " does not seem to include dex file. "
                       << "Allow oat file use. This is potentially dangerous.";
        } else {
          // We have to fallback and found original dex files - extract them from an APK.
          // Also warn about this operation because it's potentially wasteful.
          LOG(WARNING) << "Found duplicate classes, falling back to extracting from APK : "
                       << dex_location;
          LOG(WARNING) << "NOTE: This wastes RAM and hurts startup performance.";
        }
      } else {
        // TODO: We should remove this. The fact that we're here implies -Xno-dex-file-fallback
        // was set, which means that we should never fallback. If we don't have original dex
        // files, we should just fail resolution as the flag intended.
        if (!oat_file_assistant.HasOriginalDexFiles()) {
          accept_oat_file = true;
        }

        LOG(WARNING) << "Found duplicate classes, dex-file-fallback disabled, will be failing to "
                        " load classes for " << dex_location;
      }

      LOG(WARNING) << error_msg;
    }

    if (accept_oat_file) {
      VLOG(class_linker) << "Registering " << oat_file->GetLocation();
      source_oat_file = RegisterOatFile(std::move(oat_file));//这里把oat_file注册给source_oat_file
      *out_oat_file = source_oat_file;
    }
  }

  std::vector<std::unique_ptr<const DexFile>> dex_files;

  // Load the dex files from the oat file.
  if (source_oat_file != nullptr) {
    bool added_image_space = false;
    if (source_oat_file->IsExecutable()) {
      std::unique_ptr<gc::space::ImageSpace> image_space =
          kEnableAppImage ? oat_file_assistant.OpenImageSpace(source_oat_file) : nullptr;
      if (image_space != nullptr) {
        ScopedObjectAccess soa(self);
        StackHandleScope<1> hs(self);
        Handle<mirror::ClassLoader> h_loader(
            hs.NewHandle(soa.Decode<mirror::ClassLoader>(class_loader)));
        // Can not load app image without class loader.
        if (h_loader != nullptr) {
          std::string temp_error_msg;
          // Add image space has a race condition since other threads could be reading from the
          // spaces array.
          {
            ScopedThreadSuspension sts(self, kSuspended);
            gc::ScopedGCCriticalSection gcs(self,
                                            gc::kGcCauseAddRemoveAppImageSpace,
                                            gc::kCollectorTypeAddRemoveAppImageSpace);
            ScopedSuspendAll ssa("Add image space");
            runtime->GetHeap()->AddSpace(image_space.get());
          }
          {
            ScopedTrace trace2(StringPrintf("Adding image space for location %s", dex_location));
            added_image_space = runtime->GetClassLinker()->AddImageSpace(image_space.get(),
                                                                         h_loader,
                                                                         dex_elements,
                                                                         dex_location,
                                                                         /*out*/&dex_files,
                                                                         /*out*/&temp_error_msg);//最终通过AddImageSpace在堆中分配了dex_elements,dex_files等的空间
          }
          if (added_image_space) {
            // Successfully added image space to heap, release the map so that it does not get
            // freed.
            image_space.release();
          } else {
            LOG(INFO) << "Failed to add image file " << temp_error_msg;
            dex_files.clear();
            {
              ScopedThreadSuspension sts(self, kSuspended);
              gc::ScopedGCCriticalSection gcs(self,
                                              gc::kGcCauseAddRemoveAppImageSpace,
                                              gc::kCollectorTypeAddRemoveAppImageSpace);
              ScopedSuspendAll ssa("Remove image space");
              runtime->GetHeap()->RemoveSpace(image_space.get());
            }
            // Non-fatal, don't update error_msg.
          }
        }
      }
    }
    if (!added_image_space) {
      DCHECK(dex_files.empty());
      dex_files = oat_file_assistant.LoadDexFiles(*source_oat_file, dex_location);//这里通过加载source_oat_file获得dex_files
    }
    if (dex_files.empty()) {
      error_msgs->push_back("Failed to open dex files from " + source_oat_file->GetLocation());
    }
  }

  // Fall back to running out of the original dex file if we couldn't load any
  // dex_files from the oat file.
  if (dex_files.empty()) {
    if (oat_file_assistant.HasOriginalDexFiles()) {
      if (Runtime::Current()->IsDexFileFallbackEnabled()) {
        static constexpr bool kVerifyChecksum = true;
        if (!DexFile::Open(
            dex_location, dex_location, kVerifyChecksum, /*out*/ &error_msg, &dex_files)) {//如果LoadDexFiles上面没有获得dex_files,直接DexFile::Open打开加载原始的dexfile
          LOG(WARNING) << error_msg;
          error_msgs->push_back("Failed to open dex files from " + std::string(dex_location)
                                + " because: " + error_msg);
        }
      } else {
        error_msgs->push_back("Fallback mode disabled, skipping dex files.");
      }
    } else {
      error_msgs->push_back("No original dex files found for dex location "
          + std::string(dex_location));
    }
  }

  return dex_files;
}
3.进入oat_file_assistant.cc的GetBestOatFile与LoadDexFiles
先看 GetBestOatFile ,当OatFileAssistant初始化status时Status构造函数通过GetFile调用OatFile::Open打开oatfile
std::unique_ptr<OatFile> OatFileAssistant::GetBestOatFile() {
  return GetBestInfo().ReleaseFileForUse();
}

OatFileAssistant::OatFileInfo& OatFileAssistant::GetBestInfo() {
  // TODO(calin): Document the side effects of class loading when
  // running dalvikvm command line.
  if (dex_parent_writable_) {
    // If the parent of the dex file is writable it means that we can
    // create the odex file. In this case we unconditionally pick the odex
    // as the best oat file. This corresponds to the regular use case when
    // apps gets installed or when they load private, secondary dex file.
    // For apps on the system partition the odex location will not be
    // writable and thus the oat location might be more up to date.
    return odex_;
  }

  // We cannot write to the odex location. This must be a system app.

  // If the oat location is usable take it.
  if (oat_.IsUseable()) {
    return oat_;
  }

  // The oat file is not usable but the odex file might be up to date.
  // This is an indication that we are dealing with an up to date prebuilt
  // (that doesn't need relocation).
  if (odex_.Status() == kOatUpToDate) {
    return odex_;
  }

  // The oat file is not usable and the odex file is not up to date.
  // However we have access to the original dex file which means we can make
  // the oat location up to date.
  if (HasOriginalDexFiles()) {
    return oat_;
  }

  // We got into the worst situation here:
  // - the oat location is not usable
  // - the prebuild odex location is not up to date
  // - and we don't have the original dex file anymore (stripped).
  // Pick the odex if it exists, or the oat if not.
  return (odex_.Status() == kOatCannotOpen) ? oat_ : odex_;
}

OatFileAssistant::OatStatus OatFileAssistant::OatFileInfo::Status() {
  if (!status_attempted_) {
    status_attempted_ = true;
    const OatFile* file = GetFile();
    if (file == nullptr) {
      // Check to see if there is a vdex file we can make use of.
      std::string error_msg;
      std::string vdex_filename = GetVdexFilename(filename_);
      std::unique_ptr<VdexFile> vdex = VdexFile::Open(vdex_filename,
                                                      /*writeable*/false,
                                                      /*low_4gb*/false,
                                                      /*unquicken*/false,
                                                      &error_msg);//这里在Status中先打开vdex,vdex也是版本新增的,可以加快启动速度
      if (vdex == nullptr) {
        status_ = kOatCannotOpen;
        VLOG(oat) << "unable to open vdex file " << vdex_filename << ": " << error_msg;
      } else {
        if (oat_file_assistant_->DexChecksumUpToDate(*vdex, &error_msg)) {
          // The vdex file does not contain enough information to determine
          // whether it is up to date with respect to the boot image, so we
          // assume it is out of date.
          VLOG(oat) << error_msg;
          status_ = kOatBootImageOutOfDate;
        } else {
          status_ = kOatDexOutOfDate;
        }
      }
    } else {
      status_ = oat_file_assistant_->GivenOatFileStatus(*file);
      VLOG(oat) << file->GetLocation() << " is " << status_
          << " with filter " << file->GetCompilerFilter();
    }
  }
  return status_;
}

const OatFile* OatFileAssistant::OatFileInfo::GetFile() {
  CHECK(!file_released_) << "GetFile called after oat file released.";
  if (!load_attempted_) {
    load_attempted_ = true;
    if (filename_provided_) {
      std::string error_msg;
      file_.reset(OatFile::Open(filename_.c_str(),
                                filename_.c_str(),
                                nullptr,
                                nullptr,
                                oat_file_assistant_->load_executable_,
                                /*low_4gb*/false,
                                oat_file_assistant_->dex_location_.c_str(),
                                &error_msg));
      if (file_.get() == nullptr) {
        VLOG(oat) << "OatFileAssistant test for existing oat file "
          << filename_ << ": " << error_msg;
      }
    }
  }
  return file_.get();
}
再看LoadDexFiles ,先通过GetOatDexFile得到上面open的OatFile的OatDexFile,在通过OpenDexFile打开获得OatDexFile的 DexFile,这里如果有多个dex,会通过for循环获得多个dex_file,组装出 dex_files。
std::vector<std::unique_ptr<const DexFile>> OatFileAssistant::LoadDexFiles(
    const OatFile& oat_file, const char* dex_location) {
  std::vector<std::unique_ptr<const DexFile>> dex_files;

  // Load the main dex file.
  std::string error_msg;
  const OatFile::OatDexFile* oat_dex_file = oat_file.GetOatDexFile(
      dex_location, nullptr, &error_msg);//这句oat_file通过获得oat_dex_file
  if (oat_dex_file == nullptr) {
    LOG(WARNING) << error_msg;
    return std::vector<std::unique_ptr<const DexFile>>();
  }

  std::unique_ptr<const DexFile> dex_file = oat_dex_file->OpenDexFile(&error_msg);//通过oat_dex_file获得dex_file
  if (dex_file.get() == nullptr) {
    LOG(WARNING) << "Failed to open dex file from oat dex file: " << error_msg;
    return std::vector<std::unique_ptr<const DexFile>>();
  }
  dex_files.push_back(std::move(dex_file));//把dex_file放入dex_files

  // Load the rest of the multidex entries
  for (size_t i = 1; ; i++) {
    std::string multidex_dex_location = DexFile::GetMultiDexLocation(i, dex_location);
    oat_dex_file = oat_file.GetOatDexFile(multidex_dex_location.c_str(), nullptr);//如果不是一个dex,通过循环加载其他的oat_dex_file
    if (oat_dex_file == nullptr) {
      // There are no more multidex entries to load.
      break;
    }

    dex_file = oat_dex_file->OpenDexFile(&error_msg);//通过oat_dex_file的OpenDexFile获得其余dex_file
    if (dex_file.get() == nullptr) {
      LOG(WARNING) << "Failed to open dex file from oat dex file: " << error_msg;
      return std::vector<std::unique_ptr<const DexFile>>();
    }
    dex_files.push_back(std::move(dex_file));//把multidex放入dex_files
  }
  return dex_files;
}
4.再往下,就是OatFile::Open,GetOatDexFile 和OpenDexFile,这3个函数就进入了 oat_file.cc中, OatFile::Open打开oat_file比较复杂,我们下一篇在分析,假设它很愉快完成了,得到了 oat_file ,后面通过 GetOatDexFile操作它得到 oat_dex_file,在通过 OpenDexFile操作 OatDexFile最终获得 dex_file返回。这三个类互相关联, OatFile包含 OatDexFile包含 DexFile,阅读 oat_file.cc源码可以搞明白他们的数据结构差异。
4.1先看 OatFile::Open ,先试图通过OatFileBase::OpenOatFile<DlOpenOatFile> oat_file ,我们知道art分为quick和portable两中优化模式,这里先假设他是portable尝试打开oat,失败就通过OatFileBase::OpenOatFile<ElfOatFile>打开 oat_file ,这就是quick模式了,和5.0以前版本的先判断quick还是portable模式在选择函数不太一样。oat文件的加载 就在这个OpenOatFile 函数中,下一篇我们重点分析oat加载流程
OatFile* OatFile::Open(const std::string& oat_filename,
                       const std::string& oat_location,
                       uint8_t* requested_base,
                       uint8_t* oat_file_begin,
                       bool executable,
                       bool low_4gb,
                       const char* abs_dex_location,
                       std::string* error_msg) {
  ScopedTrace trace("Open oat file " + oat_location);
  CHECK(!oat_filename.empty()) << oat_location;
  CheckLocation(oat_location);

  std::string vdex_filename = GetVdexFilename(oat_filename);//首先获得vdex的信息

  // Check that the files even exist, fast-fail.
  if (kIsVdexEnabled && !OS::FileExists(vdex_filename.c_str())) {
    *error_msg = StringPrintf("File %s does not exist.", vdex_filename.c_str());
    return nullptr;
  } else if (!OS::FileExists(oat_filename.c_str())) {
    *error_msg = StringPrintf("File %s does not exist.", oat_filename.c_str());
    return nullptr;
  }

  // Try dlopen first, as it is required for native debuggability. This will fail fast if dlopen is
  // disabled.
  OatFile* with_dlopen = OatFileBase::OpenOatFile<DlOpenOatFile>(vdex_filename,
                                                                 oat_filename,
                                                                 oat_location,
                                                                 requested_base,
                                                                 oat_file_begin,
                                                                 false /* writable */,
                                                                 executable,
                                                                 low_4gb,
                                                                 abs_dex_location,
                                                                 error_msg);//先试图通过OatFileBase::OpenOatFile<DlOpenOatFile>打开OatFile,我们知道art分为quick和portable两中优化模式,这里先假设他是portable尝试打开oat
  if (with_dlopen != nullptr) {
    return with_dlopen;
  }
  if (kPrintDlOpenErrorMessage) {
    LOG(ERROR) << "Failed to dlopen: " << oat_filename << " with error " << *error_msg;
  }
  // If we aren't trying to execute, we just use our own ElfFile loader for a couple reasons:
  //
  // On target, dlopen may fail when compiling due to selinux restrictions on installd.
  //
  // We use our own ELF loader for Quick to deal with legacy apps that
  // open a generated dex file by name, remove the file, then open
  // another generated dex file with the same name. http://b/10614658
  //
  // On host, dlopen is expected to fail when cross compiling, so fall back to OpenElfFile.
  //
  //
  // Another independent reason is the absolute placement of boot.oat. dlopen on the host usually
  // does honor the virtual address encoded in the ELF file only for ET_EXEC files, not ET_DYN.
  OatFile* with_internal = OatFileBase::OpenOatFile<ElfOatFile>(vdex_filename,
                                                                oat_filename,
                                                                oat_location,
                                                                requested_base,
                                                                oat_file_begin,
                                                                false /* writable */,
                                                                executable,
                                                                low_4gb,
                                                                abs_dex_location,
                                                                error_msg);//上面尝试不成功,就通过OatFileBase::OpenOatFile<ElfOatFile>打开oat,这就是quick模式了,和以前版本加载方式不大一样,以前是先判断quick还是portable模式在选择函数。
  return with_internal;
}

OatFile* OatFile::OpenWritable(File* file,
                               const std::string& location,
                               const char* abs_dex_location,
                               std::string* error_msg) {
  CheckLocation(location);
  return ElfOatFile::OpenElfFile(file,
                                 location,
                                 nullptr,
                                 nullptr,
                                 true,
                                 false,
                                 /*low_4gb*/false,
                                 abs_dex_location,
                                 error_msg);
}
4.2后面 GetOatDexFile 通过key也就是dex_location在oat_dex_files_map里查询,从oat_file里找到oat_dex_file,这里的oat_dex_files_这个map是oat_file初始化时在setup函数中生成的,具体要看oat文件加载过程
const OatFile::OatDexFile* OatFile::GetOatDexFile(const char* dex_location,
                                                  const uint32_t* dex_location_checksum,
                                                  std::string* error_msg) const {
  // NOTE: We assume here that the canonical location for a given dex_location never
  // changes. If it does (i.e. some symlink used by the filename changes) we may return
  // an incorrect OatDexFile. As long as we have a checksum to check, we shall return
  // an identical file or fail; otherwise we may see some unpredictable failures.

  // TODO: Additional analysis of usage patterns to see if this can be simplified
  // without any performance loss, for example by not doing the first lock-free lookup.

  const OatFile::OatDexFile* oat_dex_file = nullptr;
  StringPiece key(dex_location);
  // Try to find the key cheaply in the oat_dex_files_ map which holds dex locations
  // directly mentioned in the oat file and doesn't require locking.
  auto primary_it = oat_dex_files_.find(key);//通过key也就是dex_location在oat_dex_files_ map里查询,从oat_file里找到oat_dex_file,这里的oat_dex_files_这个map是oat_file初始化时在setup函数中生成的,具体要看oat文件加载过程    
// Add the location and canonical location (if different) to the oat_dex_files_ table.
//    StringPiece key(oat_dex_file->GetDexFileLocation());
 //   oat_dex_files_.Put(key, oat_dex_file);

  if (primary_it != oat_dex_files_.end()) {//下面几个if是判断其他特殊情况,处理异常的,比如dex_location不是唯一,dex_location没有找到如何加载和报错
    oat_dex_file = primary_it->second;
    DCHECK(oat_dex_file != nullptr);
  } else {
    // This dex_location is not one of the dex locations directly mentioned in the
    // oat file. The correct lookup is via the canonical location but first see in
    // the secondary_oat_dex_files_ whether we've looked up this location before.
    MutexLock mu(Thread::Current(), secondary_lookup_lock_);
    auto secondary_lb = secondary_oat_dex_files_.lower_bound(key);
    if (secondary_lb != secondary_oat_dex_files_.end() && key == secondary_lb->first) {
      oat_dex_file = secondary_lb->second;  // May be null.
    } else {
      // We haven't seen this dex_location before, we must check the canonical location.
      std::string dex_canonical_location = DexFile::GetDexCanonicalLocation(dex_location);//如果没有找到dex_location,这里会根据绝对路径重新加载dex
      if (dex_canonical_location != dex_location) {
        StringPiece canonical_key(dex_canonical_location);
        auto canonical_it = oat_dex_files_.find(canonical_key);
        if (canonical_it != oat_dex_files_.end()) {
          oat_dex_file = canonical_it->second;
        }  // else keep null.
      }  // else keep null.

      // Copy the key to the string_cache_ and store the result in secondary map.
      string_cache_.emplace_back(key.data(), key.length());
      StringPiece key_copy(string_cache_.back());
      secondary_oat_dex_files_.PutBefore(secondary_lb, key_copy, oat_dex_file);//这里把根据绝对路径重新加载的dex信息放入secondary map,以便下次使用
    }
  }

  if (oat_dex_file == nullptr) {
    if (error_msg != nullptr) {
      std::string dex_canonical_location = DexFile::GetDexCanonicalLocation(dex_location);
      *error_msg = "Failed to find OatDexFile for DexFile " + std::string(dex_location)
          + " (canonical path " + dex_canonical_location + ") in OatFile " + GetLocation();
    }
    return nullptr;
  }

  if (dex_location_checksum != nullptr &&
      oat_dex_file->GetDexFileLocationChecksum() != *dex_location_checksum) {
    if (error_msg != nullptr) {
      std::string dex_canonical_location = DexFile::GetDexCanonicalLocation(dex_location);
      std::string checksum = StringPrintf("0x%08x", oat_dex_file->GetDexFileLocationChecksum());
      std::string required_checksum = StringPrintf("0x%08x", *dex_location_checksum);
      *error_msg = "OatDexFile for DexFile " + std::string(dex_location)
          + " (canonical path " + dex_canonical_location + ") in OatFile " + GetLocation()
          + " has checksum " + checksum + " but " + required_checksum + " was required";
    }
    return nullptr;
  }
  return oat_dex_file;
}
4.3通过GetOatDexFile 获得 oat_file 里的oat_dex_file后,就可以调用OpenDexFile获得 oat_dex_file 存储的 dex_file的信息了,这里就走到了dex_file.cc文件里的DexFile::Open这个关键函数,而这个 DexFile::Open 最终调用了OpenCommon这个关键函数,这里就是我们常用的一个脱壳函数,下一篇分析 DexFile::Open 下层的函数,就可以搞明白大佬们为什么hook OpenCommon 和OpenAndReadMagic来脱整体的dex加固
std::unique_ptr<const DexFile> OatFile::OatDexFile::OpenDexFile(std::string* error_msg) const {
  ScopedTrace trace(__PRETTY_FUNCTION__);
  static constexpr bool kVerify = false;
  static constexpr bool kVerifyChecksum = false;
  return DexFile::Open(dex_file_pointer_,
                       FileSize(),
                       dex_file_location_,
                       dex_file_location_checksum_,
                       this,
                       kVerify,
                       kVerifyChecksum,
                       error_msg);
}
总结一下,时间仓促,有些注释可能有点错误,有些细节实现我也没仔细看,希望大家指出我的错误我好努力改进,在libre里画个及其丑陋的图辅助自己理解,箭头指向被调用的函数,双向箭头两个函数再同一个上层函数里先后调用,从opeDexFileNative最终调用到了OatFile::Open与DexFile::Open ,前者获得oat_file,,后者通过获得的oat_file得数据结构获得的 oat_dex_file的数据结构获得dex_file,下一篇菜鸟整理一下这2个函数是如何获得 oat_file与 oat_dex_file并加载到内存中的。





static jobject DexFile_openDexFileNative(JNIEnv* env,
                                         jclass,
                                         jstring javaSourceName,
                                         jstring javaOutputName ATTRIBUTE_UNUSED,
                                         jint flags ATTRIBUTE_UNUSED,
                                         jobject class_loader,
                                         jobjectArray dex_elements) {
  ScopedUtfChars sourceName(env, javaSourceName);
  if (sourceName.c_str() == nullptr) {
    return 0;
  }

  Runtime* const runtime = Runtime::Current();
  ClassLinker* linker = runtime->GetClassLinker(); //获得linker
  std::vector<std::unique_ptr<const DexFile>> dex_files;
  std::vector<std::string> error_msgs;
  const OatFile* oat_file = nullptr;

  dex_files = runtime->GetOatFileManager().OpenDexFilesFromOat(sourceName.c_str(),
                                                               class_loader,
                                                               dex_elements,
                                                               /*out*/ &oat_file,
                                                               /*out*/ &error_msgs);//这句调用OpenDexFilesFromOat获得dex_files

  if (!dex_files.empty()) {
    jlongArray array = ConvertDexFilesToJavaArray(env, oat_file, dex_files);//这句把DexFiles转化为JavaArray,方便java层使用
    if (array == nullptr) {
      ScopedObjectAccess soa(env);
      for (auto& dex_file : dex_files) {
        if (linker->IsDexFileRegistered(soa.Self(), *dex_file)) {  //这里linker只用来判断dex_file是不是已经存在,跟以前版本不同,需要注意一下
          dex_file.release();
        }
      }
    }
    return array;
  } else {
    ScopedObjectAccess soa(env);
    CHECK(!error_msgs.empty());
    // The most important message is at the end. So set up nesting by going forward, which will
    // wrap the existing exception as a cause for the following one.
    auto it = error_msgs.begin();
    auto itEnd = error_msgs.end();
    for ( ; it != itEnd; ++it) {
      ThrowWrappedIOException("%s", it->c_str());
    }

    return nullptr;
  }
}

2.进入oat_file_manager.cc的 OpenDexFilesFromOat

2.1,先通过:std::unique_ptr<const OatFile> oat_file(oat_file_assistant.GetBestOatFile().release())这句获得了oat_file,

2.2然后通过:dex_files = oat_file_assistant.LoadDexFiles(*source_oat_file, dex_location)这里通过加载source_oat_file获得dex_files

2.3如果上面2.1与2.2不成立:DexFile::Open(dex_location, dex_location, kVerifyChecksum, /*out*/ &error_msg, &dex_files)) //如果LoadDexFiles上面没有获得dex_files,直接DexFile::Open打开加载原始的dexfile

std::vector<std::unique_ptr<const DexFile>> OatFileManager::OpenDexFilesFromOat(
    const char* dex_location,
    jobject class_loader,
    jobjectArray dex_elements,
    const OatFile** out_oat_file,
    std::vector<std::string>* error_msgs) {
  ScopedTrace trace(__FUNCTION__);
  CHECK(dex_location != nullptr);
  CHECK(error_msgs != nullptr);

  // Verify we aren't holding the mutator lock, which could starve GC if we
  // have to generate or relocate an oat file.
  Thread* const self = Thread::Current();
  Locks::mutator_lock_->AssertNotHeld(self);
  Runtime* const runtime = Runtime::Current();

  OatFileAssistant oat_file_assistant(dex_location,
                                      kRuntimeISA,
                                      !runtime->IsAotCompiler());

  // Lock the target oat location to avoid races generating and loading the
  // oat file.
  std::string error_msg;
  if (!oat_file_assistant.Lock(/*out*/&error_msg)) {
    // Don't worry too much if this fails. If it does fail, it's unlikely we
    // can generate an oat file anyway.
    VLOG(class_linker) << "OatFileAssistant::Lock: " << error_msg;
  }

  const OatFile* source_oat_file = nullptr;

  if (!oat_file_assistant.IsUpToDate()) {
    // Update the oat file on disk if we can, based on the --compiler-filter
    // option derived from the current runtime options.
    // This may fail, but that's okay. Best effort is all that matters here.
    switch (oat_file_assistant.MakeUpToDate(/*profile_changed*/false, /*out*/ &error_msg)) {
      case OatFileAssistant::kUpdateFailed:
        LOG(WARNING) << error_msg;
        break;

      case OatFileAssistant::kUpdateNotAttempted:
        // Avoid spamming the logs if we decided not to attempt making the oat
        // file up to date.
        VLOG(oat) << error_msg;
        break;

      case OatFileAssistant::kUpdateSucceeded:
        // Nothing to do.
        break;
    }
  }

  // Get the oat file on disk.
  std::unique_ptr<const OatFile> oat_file(oat_file_assistant.GetBestOatFile().release());//这句获得了oat_file,下面LoadDexFiles使用这个oat_file获得dex_files

  if (oat_file != nullptr) {
    // Take the file only if it has no collisions, or we must take it because of preopting.
    bool accept_oat_file =
        !HasCollisions(oat_file.get(), class_loader, dex_elements, /*out*/ &error_msg);
    if (!accept_oat_file) {
      // Failed the collision check. Print warning.
      if (Runtime::Current()->IsDexFileFallbackEnabled()) {
        if (!oat_file_assistant.HasOriginalDexFiles()) {
          // We need to fallback but don't have original dex files. We have to
          // fallback to opening the existing oat file. This is potentially
          // unsafe so we warn about it.
          accept_oat_file = true;

          LOG(WARNING) << "Dex location " << dex_location << " does not seem to include dex file. "
                       << "Allow oat file use. This is potentially dangerous.";
        } else {
          // We have to fallback and found original dex files - extract them from an APK.
          // Also warn about this operation because it's potentially wasteful.
          LOG(WARNING) << "Found duplicate classes, falling back to extracting from APK : "
                       << dex_location;
          LOG(WARNING) << "NOTE: This wastes RAM and hurts startup performance.";
        }
      } else {
        // TODO: We should remove this. The fact that we're here implies -Xno-dex-file-fallback
        // was set, which means that we should never fallback. If we don't have original dex
        // files, we should just fail resolution as the flag intended.
        if (!oat_file_assistant.HasOriginalDexFiles()) {
          accept_oat_file = true;
        }

        LOG(WARNING) << "Found duplicate classes, dex-file-fallback disabled, will be failing to "
                        " load classes for " << dex_location;
      }

      LOG(WARNING) << error_msg;
    }

    if (accept_oat_file) {
      VLOG(class_linker) << "Registering " << oat_file->GetLocation();
      source_oat_file = RegisterOatFile(std::move(oat_file));//这里把oat_file注册给source_oat_file
      *out_oat_file = source_oat_file;
    }
  }

  std::vector<std::unique_ptr<const DexFile>> dex_files;

  // Load the dex files from the oat file.
  if (source_oat_file != nullptr) {
    bool added_image_space = false;
    if (source_oat_file->IsExecutable()) {
      std::unique_ptr<gc::space::ImageSpace> image_space =
          kEnableAppImage ? oat_file_assistant.OpenImageSpace(source_oat_file) : nullptr;
      if (image_space != nullptr) {
        ScopedObjectAccess soa(self);
        StackHandleScope<1> hs(self);
        Handle<mirror::ClassLoader> h_loader(
            hs.NewHandle(soa.Decode<mirror::ClassLoader>(class_loader)));
        // Can not load app image without class loader.
        if (h_loader != nullptr) {
          std::string temp_error_msg;
          // Add image space has a race condition since other threads could be reading from the
          // spaces array.
          {
            ScopedThreadSuspension sts(self, kSuspended);
            gc::ScopedGCCriticalSection gcs(self,
                                            gc::kGcCauseAddRemoveAppImageSpace,
                                            gc::kCollectorTypeAddRemoveAppImageSpace);
            ScopedSuspendAll ssa("Add image space");
            runtime->GetHeap()->AddSpace(image_space.get());
          }
          {
            ScopedTrace trace2(StringPrintf("Adding image space for location %s", dex_location));
            added_image_space = runtime->GetClassLinker()->AddImageSpace(image_space.get(),
                                                                         h_loader,
                                                                         dex_elements,
                                                                         dex_location,
                                                                         /*out*/&dex_files,
                                                                         /*out*/&temp_error_msg);//最终通过AddImageSpace在堆中分配了dex_elements,dex_files等的空间
          }
          if (added_image_space) {
            // Successfully added image space to heap, release the map so that it does not get
            // freed.
            image_space.release();
          } else {
            LOG(INFO) << "Failed to add image file " << temp_error_msg;
            dex_files.clear();
            {
              ScopedThreadSuspension sts(self, kSuspended);
              gc::ScopedGCCriticalSection gcs(self,
                                              gc::kGcCauseAddRemoveAppImageSpace,
                                              gc::kCollectorTypeAddRemoveAppImageSpace);
              ScopedSuspendAll ssa("Remove image space");
              runtime->GetHeap()->RemoveSpace(image_space.get());
            }
            // Non-fatal, don't update error_msg.
          }
        }
      }
    }
    if (!added_image_space) {
      DCHECK(dex_files.empty());
      dex_files = oat_file_assistant.LoadDexFiles(*source_oat_file, dex_location);//这里通过加载source_oat_file获得dex_files
    }
    if (dex_files.empty()) {
      error_msgs->push_back("Failed to open dex files from " + source_oat_file->GetLocation());
    }
  }

  // Fall back to running out of the original dex file if we couldn't load any
  // dex_files from the oat file.
  if (dex_files.empty()) {
    if (oat_file_assistant.HasOriginalDexFiles()) {
      if (Runtime::Current()->IsDexFileFallbackEnabled()) {
        static constexpr bool kVerifyChecksum = true;
        if (!DexFile::Open(
            dex_location, dex_location, kVerifyChecksum, /*out*/ &error_msg, &dex_files)) {//如果LoadDexFiles上面没有获得dex_files,直接DexFile::Open打开加载原始的dexfile
          LOG(WARNING) << error_msg;
          error_msgs->push_back("Failed to open dex files from " + std::string(dex_location)
                                + " because: " + error_msg);
        }
      } else {
        error_msgs->push_back("Fallback mode disabled, skipping dex files.");
      }
    } else {
      error_msgs->push_back("No original dex files found for dex location "
          + std::string(dex_location));
    }
  }

  return dex_files;
}
3.进入oat_file_assistant.cc的GetBestOatFile与LoadDexFiles
先看 GetBestOatFile ,当OatFileAssistant初始化status时Status构造函数通过GetFile调用OatFile::Open打开oatfile
std::unique_ptr<OatFile> OatFileAssistant::GetBestOatFile() {
  return GetBestInfo().ReleaseFileForUse();
}

OatFileAssistant::OatFileInfo& OatFileAssistant::GetBestInfo() {
  // TODO(calin): Document the side effects of class loading when
  // running dalvikvm command line.
  if (dex_parent_writable_) {
    // If the parent of the dex file is writable it means that we can
    // create the odex file. In this case we unconditionally pick the odex
    // as the best oat file. This corresponds to the regular use case when
    // apps gets installed or when they load private, secondary dex file.
    // For apps on the system partition the odex location will not be
    // writable and thus the oat location might be more up to date.
    return odex_;
  }

  // We cannot write to the odex location. This must be a system app.

  // If the oat location is usable take it.
  if (oat_.IsUseable()) {
    return oat_;
  }

  // The oat file is not usable but the odex file might be up to date.
  // This is an indication that we are dealing with an up to date prebuilt
  // (that doesn't need relocation).
  if (odex_.Status() == kOatUpToDate) {
    return odex_;
  }

  // The oat file is not usable and the odex file is not up to date.
  // However we have access to the original dex file which means we can make
  // the oat location up to date.
  if (HasOriginalDexFiles()) {
    return oat_;
  }

  // We got into the worst situation here:
  // - the oat location is not usable
  // - the prebuild odex location is not up to date
  // - and we don't have the original dex file anymore (stripped).
  // Pick the odex if it exists, or the oat if not.
  return (odex_.Status() == kOatCannotOpen) ? oat_ : odex_;
}

OatFileAssistant::OatStatus OatFileAssistant::OatFileInfo::Status() {
  if (!status_attempted_) {
    status_attempted_ = true;
    const OatFile* file = GetFile();
    if (file == nullptr) {
      // Check to see if there is a vdex file we can make use of.
      std::string error_msg;
      std::string vdex_filename = GetVdexFilename(filename_);
      std::unique_ptr<VdexFile> vdex = VdexFile::Open(vdex_filename,
                                                      /*writeable*/false,
                                                      /*low_4gb*/false,
                                                      /*unquicken*/false,
                                                      &error_msg);//这里在Status中先打开vdex,vdex也是版本新增的,可以加快启动速度
      if (vdex == nullptr) {
        status_ = kOatCannotOpen;
        VLOG(oat) << "unable to open vdex file " << vdex_filename << ": " << error_msg;
      } else {
        if (oat_file_assistant_->DexChecksumUpToDate(*vdex, &error_msg)) {
          // The vdex file does not contain enough information to determine
          // whether it is up to date with respect to the boot image, so we
          // assume it is out of date.
          VLOG(oat) << error_msg;
          status_ = kOatBootImageOutOfDate;
        } else {
          status_ = kOatDexOutOfDate;
        }
      }
    } else {
      status_ = oat_file_assistant_->GivenOatFileStatus(*file);
      VLOG(oat) << file->GetLocation() << " is " << status_
          << " with filter " << file->GetCompilerFilter();
    }
  }
  return status_;
}

const OatFile* OatFileAssistant::OatFileInfo::GetFile() {
  CHECK(!file_released_) << "GetFile called after oat file released.";
  if (!load_attempted_) {
    load_attempted_ = true;
    if (filename_provided_) {
      std::string error_msg;
      file_.reset(OatFile::Open(filename_.c_str(),
                                filename_.c_str(),
                                nullptr,
                                nullptr,
                                oat_file_assistant_->load_executable_,
                                /*low_4gb*/false,
                                oat_file_assistant_->dex_location_.c_str(),
                                &error_msg));
      if (file_.get() == nullptr) {
        VLOG(oat) << "OatFileAssistant test for existing oat file "
          << filename_ << ": " << error_msg;
      }
    }
  }
  return file_.get();
}
再看LoadDexFiles ,先通过GetOatDexFile得到上面open的OatFile的OatDexFile,在通过OpenDexFile打开获得OatDexFile的 DexFile,这里如果有多个dex,会通过for循环获得多个dex_file,组装出 dex_files。
std::vector<std::unique_ptr<const DexFile>> OatFileAssistant::LoadDexFiles(
    const OatFile& oat_file, const char* dex_location) {
  std::vector<std::unique_ptr<const DexFile>> dex_files;

  // Load the main dex file.
  std::string error_msg;
  const OatFile::OatDexFile* oat_dex_file = oat_file.GetOatDexFile(
      dex_location, nullptr, &error_msg);//这句oat_file通过获得oat_dex_file
  if (oat_dex_file == nullptr) {
    LOG(WARNING) << error_msg;
    return std::vector<std::unique_ptr<const DexFile>>();
  }

  std::unique_ptr<const DexFile> dex_file = oat_dex_file->OpenDexFile(&error_msg);//通过oat_dex_file获得dex_file
  if (dex_file.get() == nullptr) {
    LOG(WARNING) << "Failed to open dex file from oat dex file: " << error_msg;
    return std::vector<std::unique_ptr<const DexFile>>();
  }
  dex_files.push_back(std::move(dex_file));//把dex_file放入dex_files

  // Load the rest of the multidex entries
  for (size_t i = 1; ; i++) {
    std::string multidex_dex_location = DexFile::GetMultiDexLocation(i, dex_location);
    oat_dex_file = oat_file.GetOatDexFile(multidex_dex_location.c_str(), nullptr);//如果不是一个dex,通过循环加载其他的oat_dex_file
    if (oat_dex_file == nullptr) {
      // There are no more multidex entries to load.
      break;
    }

    dex_file = oat_dex_file->OpenDexFile(&error_msg);//通过oat_dex_file的OpenDexFile获得其余dex_file
    if (dex_file.get() == nullptr) {
      LOG(WARNING) << "Failed to open dex file from oat dex file: " << error_msg;
      return std::vector<std::unique_ptr<const DexFile>>();
    }
    dex_files.push_back(std::move(dex_file));//把multidex放入dex_files
  }
  return dex_files;
}
4.再往下,就是OatFile::Open,GetOatDexFile 和OpenDexFile,这3个函数就进入了 oat_file.cc中, OatFile::Open打开oat_file比较复杂,我们下一篇在分析,假设它很愉快完成了,得到了 oat_file ,后面通过 GetOatDexFile操作它得到 oat_dex_file,在通过 OpenDexFile操作 OatDexFile最终获得 dex_file返回。这三个类互相关联, OatFile包含 OatDexFile包含 DexFile,阅读 oat_file.cc源码可以搞明白他们的数据结构差异。
4.1先看 OatFile::Open ,先试图通过OatFileBase::OpenOatFile<DlOpenOatFile> oat_file ,我们知道art分为quick和portable两中优化模式,这里先假设他是portable尝试打开oat,失败就通过OatFileBase::OpenOatFile<ElfOatFile>打开 oat_file ,这就是quick模式了,和5.0以前版本的先判断quick还是portable模式在选择函数不太一样。oat文件的加载 就在这个OpenOatFile 函数中,下一篇我们重点分析oat加载流程
OatFile* OatFile::Open(const std::string& oat_filename,
                       const std::string& oat_location,
                       uint8_t* requested_base,
                       uint8_t* oat_file_begin,
                       bool executable,
                       bool low_4gb,
                       const char* abs_dex_location,
                       std::string* error_msg) {
  ScopedTrace trace("Open oat file " + oat_location);
  CHECK(!oat_filename.empty()) << oat_location;
  CheckLocation(oat_location);

  std::string vdex_filename = GetVdexFilename(oat_filename);//首先获得vdex的信息

  // Check that the files even exist, fast-fail.
  if (kIsVdexEnabled && !OS::FileExists(vdex_filename.c_str())) {
    *error_msg = StringPrintf("File %s does not exist.", vdex_filename.c_str());
    return nullptr;
  } else if (!OS::FileExists(oat_filename.c_str())) {
    *error_msg = StringPrintf("File %s does not exist.", oat_filename.c_str());
    return nullptr;
  }

  // Try dlopen first, as it is required for native debuggability. This will fail fast if dlopen is
  // disabled.
  OatFile* with_dlopen = OatFileBase::OpenOatFile<DlOpenOatFile>(vdex_filename,
                                                                 oat_filename,
                                                                 oat_location,
                                                                 requested_base,
                                                                 oat_file_begin,
                                                                 false /* writable */,
                                                                 executable,
                                                                 low_4gb,
                                                                 abs_dex_location,
                                                                 error_msg);//先试图通过OatFileBase::OpenOatFile<DlOpenOatFile>打开OatFile,我们知道art分为quick和portable两中优化模式,这里先假设他是portable尝试打开oat
  if (with_dlopen != nullptr) {
    return with_dlopen;
  }
  if (kPrintDlOpenErrorMessage) {
    LOG(ERROR) << "Failed to dlopen: " << oat_filename << " with error " << *error_msg;
  }
  // If we aren't trying to execute, we just use our own ElfFile loader for a couple reasons:
  //
  // On target, dlopen may fail when compiling due to selinux restrictions on installd.
  //
  // We use our own ELF loader for Quick to deal with legacy apps that
  // open a generated dex file by name, remove the file, then open
  // another generated dex file with the same name. http://b/10614658
  //
  // On host, dlopen is expected to fail when cross compiling, so fall back to OpenElfFile.
  //
  //
  // Another independent reason is the absolute placement of boot.oat. dlopen on the host usually
  // does honor the virtual address encoded in the ELF file only for ET_EXEC files, not ET_DYN.
  OatFile* with_internal = OatFileBase::OpenOatFile<ElfOatFile>(vdex_filename,
                                                                oat_filename,
                                                                oat_location,
                                                                requested_base,
                                                                oat_file_begin,
                                                                false /* writable */,
                                                                executable,
                                                                low_4gb,
                                                                abs_dex_location,
                                                                error_msg);//上面尝试不成功,就通过OatFileBase::OpenOatFile<ElfOatFile>打开oat,这就是quick模式了,和以前版本加载方式不大一样,以前是先判断quick还是portable模式在选择函数。
  return with_internal;
}

OatFile* OatFile::OpenWritable(File* file,
                               const std::string& location,
                               const char* abs_dex_location,
                               std::string* error_msg) {
  CheckLocation(location);
  return ElfOatFile::OpenElfFile(file,
                                 location,
                                 nullptr,
                                 nullptr,
                                 true,
                                 false,
                                 /*low_4gb*/false,
                                 abs_dex_location,
                                 error_msg);
}
4.2后面 GetOatDexFile 通过key也就是dex_location在oat_dex_files_map里查询,从oat_file里找到oat_dex_file,这里的oat_dex_files_这个map是oat_file初始化时在setup函数中生成的,具体要看oat文件加载过程
const OatFile::OatDexFile* OatFile::GetOatDexFile(const char* dex_location,
                                                  const uint32_t* dex_location_checksum,
                                                  std::string* error_msg) const {
  // NOTE: We assume here that the canonical location for a given dex_location never
  // changes. If it does (i.e. some symlink used by the filename changes) we may return
  // an incorrect OatDexFile. As long as we have a checksum to check, we shall return
  // an identical file or fail; otherwise we may see some unpredictable failures.

  // TODO: Additional analysis of usage patterns to see if this can be simplified
  // without any performance loss, for example by not doing the first lock-free lookup.

  const OatFile::OatDexFile* oat_dex_file = nullptr;
  StringPiece key(dex_location);
  // Try to find the key cheaply in the oat_dex_files_ map which holds dex locations
  // directly mentioned in the oat file and doesn't require locking.
  auto primary_it = oat_dex_files_.find(key);//通过key也就是dex_location在oat_dex_files_ map里查询,从oat_file里找到oat_dex_file,这里的oat_dex_files_这个map是oat_file初始化时在setup函数中生成的,具体要看oat文件加载过程    
// Add the location and canonical location (if different) to the oat_dex_files_ table.
//    StringPiece key(oat_dex_file->GetDexFileLocation());
 //   oat_dex_files_.Put(key, oat_dex_file);

  if (primary_it != oat_dex_files_.end()) {//下面几个if是判断其他特殊情况,处理异常的,比如dex_location不是唯一,dex_location没有找到如何加载和报错
    oat_dex_file = primary_it->second;
    DCHECK(oat_dex_file != nullptr);
  } else {
    // This dex_location is not one of the dex locations directly mentioned in the
    // oat file. The correct lookup is via the canonical location but first see in
    // the secondary_oat_dex_files_ whether we've looked up this location before.
    MutexLock mu(Thread::Current(), secondary_lookup_lock_);
    auto secondary_lb = secondary_oat_dex_files_.lower_bound(key);
    if (secondary_lb != secondary_oat_dex_files_.end() && key == secondary_lb->first) {
      oat_dex_file = secondary_lb->second;  // May be null.
    } else {
      // We haven't seen this dex_location before, we must check the canonical location.
      std::string dex_canonical_location = DexFile::GetDexCanonicalLocation(dex_location);//如果没有找到dex_location,这里会根据绝对路径重新加载dex
      if (dex_canonical_location != dex_location) {
        StringPiece canonical_key(dex_canonical_location);
        auto canonical_it = oat_dex_files_.find(canonical_key);
        if (canonical_it != oat_dex_files_.end()) {
          oat_dex_file = canonical_it->second;
        }  // else keep null.
      }  // else keep null.

      // Copy the key to the string_cache_ and store the result in secondary map.
      string_cache_.emplace_back(key.data(), key.length());
      StringPiece key_copy(string_cache_.back());
      secondary_oat_dex_files_.PutBefore(secondary_lb, key_copy, oat_dex_file);//这里把根据绝对路径重新加载的dex信息放入secondary map,以便下次使用
    }
  }

  if (oat_dex_file == nullptr) {
    if (error_msg != nullptr) {
      std::string dex_canonical_location = DexFile::GetDexCanonicalLocation(dex_location);
      *error_msg = "Failed to find OatDexFile for DexFile " + std::string(dex_location)
          + " (canonical path " + dex_canonical_location + ") in OatFile " + GetLocation();
    }
    return nullptr;
  }

  if (dex_location_checksum != nullptr &&
      oat_dex_file->GetDexFileLocationChecksum() != *dex_location_checksum) {
    if (error_msg != nullptr) {
      std::string dex_canonical_location = DexFile::GetDexCanonicalLocation(dex_location);
      std::string checksum = StringPrintf("0x%08x", oat_dex_file->GetDexFileLocationChecksum());
      std::string required_checksum = StringPrintf("0x%08x", *dex_location_checksum);
      *error_msg = "OatDexFile for DexFile " + std::string(dex_location)
          + " (canonical path " + dex_canonical_location + ") in OatFile " + GetLocation()
          + " has checksum " + checksum + " but " + required_checksum + " was required";
    }
    return nullptr;
  }
  return oat_dex_file;
}
4.3通过GetOatDexFile 获得 oat_file 里的oat_dex_file后,就可以调用OpenDexFile获得 oat_dex_file 存储的 dex_file的信息了,这里就走到了dex_file.cc文件里的DexFile::Open这个关键函数,而这个 DexFile::Open 最终调用了OpenCommon这个关键函数,这里就是我们常用的一个脱壳函数,下一篇分析 DexFile::Open 下层的函数,就可以搞明白大佬们为什么hook OpenCommon 和OpenAndReadMagic来脱整体的dex加固
std::unique_ptr<const DexFile> OatFile::OatDexFile::OpenDexFile(std::string* error_msg) const {
  ScopedTrace trace(__PRETTY_FUNCTION__);
  static constexpr bool kVerify = false;
  static constexpr bool kVerifyChecksum = false;
  return DexFile::Open(dex_file_pointer_,
                       FileSize(),
                       dex_file_location_,
                       dex_file_location_checksum_,
                       this,
                       kVerify,
                       kVerifyChecksum,
                       error_msg);
}
总结一下,时间仓促,有些注释可能有点错误,有些细节实现我也没仔细看,希望大家指出我的错误我好努力改进,在libre里画个及其丑陋的图辅助自己理解,箭头指向被调用的函数,双向箭头两个函数再同一个上层函数里先后调用,从opeDexFileNative最终调用到了OatFile::Open与DexFile::Open ,前者获得oat_file,,后者通过获得的oat_file得数据结构获得的 oat_dex_file的数据结构获得dex_file,下一篇菜鸟整理一下这2个函数是如何获得 oat_file与 oat_dex_file并加载到内存中的。





std::vector<std::unique_ptr<const DexFile>> OatFileManager::OpenDexFilesFromOat(
    const char* dex_location,
    jobject class_loader,
    jobjectArray dex_elements,
    const OatFile** out_oat_file,
    std::vector<std::string>* error_msgs) {
  ScopedTrace trace(__FUNCTION__);
  CHECK(dex_location != nullptr);
  CHECK(error_msgs != nullptr);

  // Verify we aren't holding the mutator lock, which could starve GC if we
  // have to generate or relocate an oat file.
  Thread* const self = Thread::Current();
  Locks::mutator_lock_->AssertNotHeld(self);
  Runtime* const runtime = Runtime::Current();

  OatFileAssistant oat_file_assistant(dex_location,
                                      kRuntimeISA,
                                      !runtime->IsAotCompiler());

  // Lock the target oat location to avoid races generating and loading the
  // oat file.
  std::string error_msg;
  if (!oat_file_assistant.Lock(/*out*/&error_msg)) {
    // Don't worry too much if this fails. If it does fail, it's unlikely we
    // can generate an oat file anyway.
    VLOG(class_linker) << "OatFileAssistant::Lock: " << error_msg;
  }

  const OatFile* source_oat_file = nullptr;

  if (!oat_file_assistant.IsUpToDate()) {
    // Update the oat file on disk if we can, based on the --compiler-filter
    // option derived from the current runtime options.
    // This may fail, but that's okay. Best effort is all that matters here.
    switch (oat_file_assistant.MakeUpToDate(/*profile_changed*/false, /*out*/ &error_msg)) {
      case OatFileAssistant::kUpdateFailed:
        LOG(WARNING) << error_msg;
        break;

      case OatFileAssistant::kUpdateNotAttempted:
        // Avoid spamming the logs if we decided not to attempt making the oat
        // file up to date.
        VLOG(oat) << error_msg;
        break;

      case OatFileAssistant::kUpdateSucceeded:
        // Nothing to do.
        break;
    }
  }

  // Get the oat file on disk.
  std::unique_ptr<const OatFile> oat_file(oat_file_assistant.GetBestOatFile().release());//这句获得了oat_file,下面LoadDexFiles使用这个oat_file获得dex_files

  if (oat_file != nullptr) {
    // Take the file only if it has no collisions, or we must take it because of preopting.
    bool accept_oat_file =
        !HasCollisions(oat_file.get(), class_loader, dex_elements, /*out*/ &error_msg);
    if (!accept_oat_file) {
      // Failed the collision check. Print warning.
      if (Runtime::Current()->IsDexFileFallbackEnabled()) {
        if (!oat_file_assistant.HasOriginalDexFiles()) {
          // We need to fallback but don't have original dex files. We have to
          // fallback to opening the existing oat file. This is potentially
          // unsafe so we warn about it.
          accept_oat_file = true;

          LOG(WARNING) << "Dex location " << dex_location << " does not seem to include dex file. "
                       << "Allow oat file use. This is potentially dangerous.";
        } else {
          // We have to fallback and found original dex files - extract them from an APK.
          // Also warn about this operation because it's potentially wasteful.
          LOG(WARNING) << "Found duplicate classes, falling back to extracting from APK : "
                       << dex_location;
          LOG(WARNING) << "NOTE: This wastes RAM and hurts startup performance.";
        }
      } else {
        // TODO: We should remove this. The fact that we're here implies -Xno-dex-file-fallback
        // was set, which means that we should never fallback. If we don't have original dex
        // files, we should just fail resolution as the flag intended.
        if (!oat_file_assistant.HasOriginalDexFiles()) {
          accept_oat_file = true;
        }

        LOG(WARNING) << "Found duplicate classes, dex-file-fallback disabled, will be failing to "
                        " load classes for " << dex_location;
      }

      LOG(WARNING) << error_msg;
    }

    if (accept_oat_file) {
      VLOG(class_linker) << "Registering " << oat_file->GetLocation();
      source_oat_file = RegisterOatFile(std::move(oat_file));//这里把oat_file注册给source_oat_file
      *out_oat_file = source_oat_file;
    }
  }

  std::vector<std::unique_ptr<const DexFile>> dex_files;

  // Load the dex files from the oat file.
  if (source_oat_file != nullptr) {
    bool added_image_space = false;
    if (source_oat_file->IsExecutable()) {
      std::unique_ptr<gc::space::ImageSpace> image_space =
          kEnableAppImage ? oat_file_assistant.OpenImageSpace(source_oat_file) : nullptr;
      if (image_space != nullptr) {
        ScopedObjectAccess soa(self);
        StackHandleScope<1> hs(self);
        Handle<mirror::ClassLoader> h_loader(
            hs.NewHandle(soa.Decode<mirror::ClassLoader>(class_loader)));
        // Can not load app image without class loader.
        if (h_loader != nullptr) {
          std::string temp_error_msg;
          // Add image space has a race condition since other threads could be reading from the
          // spaces array.
          {
            ScopedThreadSuspension sts(self, kSuspended);
            gc::ScopedGCCriticalSection gcs(self,
                                            gc::kGcCauseAddRemoveAppImageSpace,
                                            gc::kCollectorTypeAddRemoveAppImageSpace);
            ScopedSuspendAll ssa("Add image space");
            runtime->GetHeap()->AddSpace(image_space.get());
          }
          {
            ScopedTrace trace2(StringPrintf("Adding image space for location %s", dex_location));
            added_image_space = runtime->GetClassLinker()->AddImageSpace(image_space.get(),
                                                                         h_loader,
                                                                         dex_elements,
                                                                         dex_location,
                                                                         /*out*/&dex_files,
                                                                         /*out*/&temp_error_msg);//最终通过AddImageSpace在堆中分配了dex_elements,dex_files等的空间
          }
          if (added_image_space) {
            // Successfully added image space to heap, release the map so that it does not get
            // freed.
            image_space.release();
          } else {
            LOG(INFO) << "Failed to add image file " << temp_error_msg;
            dex_files.clear();
            {
              ScopedThreadSuspension sts(self, kSuspended);
              gc::ScopedGCCriticalSection gcs(self,
                                              gc::kGcCauseAddRemoveAppImageSpace,
                                              gc::kCollectorTypeAddRemoveAppImageSpace);
              ScopedSuspendAll ssa("Remove image space");
              runtime->GetHeap()->RemoveSpace(image_space.get());
            }
            // Non-fatal, don't update error_msg.
          }
        }
      }
    }
    if (!added_image_space) {
      DCHECK(dex_files.empty());
      dex_files = oat_file_assistant.LoadDexFiles(*source_oat_file, dex_location);//这里通过加载source_oat_file获得dex_files
    }
    if (dex_files.empty()) {
      error_msgs->push_back("Failed to open dex files from " + source_oat_file->GetLocation());
    }
  }

  // Fall back to running out of the original dex file if we couldn't load any
  // dex_files from the oat file.
  if (dex_files.empty()) {
    if (oat_file_assistant.HasOriginalDexFiles()) {
      if (Runtime::Current()->IsDexFileFallbackEnabled()) {
        static constexpr bool kVerifyChecksum = true;
        if (!DexFile::Open(
            dex_location, dex_location, kVerifyChecksum, /*out*/ &error_msg, &dex_files)) {//如果LoadDexFiles上面没有获得dex_files,直接DexFile::Open打开加载原始的dexfile
          LOG(WARNING) << error_msg;
          error_msgs->push_back("Failed to open dex files from " + std::string(dex_location)
                                + " because: " + error_msg);
        }
      } else {
        error_msgs->push_back("Fallback mode disabled, skipping dex files.");
      }
    } else {
      error_msgs->push_back("No original dex files found for dex location "
          + std::string(dex_location));
    }
  }

  return dex_files;
}

[招生]科锐逆向工程师培训(2024年11月15日实地,远程教学同时开班, 第51期)

最后于 2020-3-28 17:36 被挤蹭菌衣编辑 ,原因:
上传的附件:
收藏
免费 8
支持
分享
最新回复 (8)
雪    币: 5235
活跃值: (3260)
能力值: ( LV10,RANK:175 )
在线值:
发帖
回帖
粉丝
2
感谢斑竹大佬的优秀
2020-3-1 10:22
0
雪    币: 3040
活跃值: (1150)
能力值: ( LV8,RANK:120 )
在线值:
发帖
回帖
粉丝
3
2020-3-1 11:19
0
雪    币: 17
活跃值: (891)
能力值: ( LV2,RANK:10 )
在线值:
发帖
回帖
粉丝
4
E4A写的APK 有个com.e4a.runtime.加密解密类 是final修饰都是静态方法 在安卓5.1用 loadclass 这个方法HOOK 是会加载的 在安卓6以上测试唯独这个类不会出现在loadclass 楼主知道是为什么吗
2020-3-2 09:26
0
雪    币: 5235
活跃值: (3260)
能力值: ( LV10,RANK:175 )
在线值:
发帖
回帖
粉丝
5
e语言没接触过哦
2020-3-2 11:16
0
雪    币:
能力值: ( LV1,RANK:0 )
在线值:
发帖
回帖
粉丝
6
不错不错
2020-3-6 12:03
0
雪    币: 2510
能力值: ( LV1,RANK:0 )
在线值:
发帖
回帖
粉丝
7
感谢分享
2020-3-11 12:53
0
雪    币: 1657
活跃值: (924)
能力值: ( LV3,RANK:20 )
在线值:
发帖
回帖
粉丝
8
支持大佬
2020-3-28 19:26
0
雪    币: 576
活跃值: (2035)
能力值: ( LV2,RANK:10 )
在线值:
发帖
回帖
粉丝
9
感谢分享
2021-6-18 21:39
0
游客
登录 | 注册 方可回帖
返回
//