Android——APK 在32bit/64bit平台 動態庫問題

目前64bit android系統也慢慢的多了，看到也有apk聲稱支持64bit system，然後就往裡面打包搞了個arm64-v8a 目錄，放了個64bit的so，但是apk代碼裡面卻不按規范去load so ，導致一系列 file not found 異常~

apk lib目錄:

先看下apk中的lib打包的目錄：

依次代表不同類型的cpu<喎?/kf/ware/vc/" target="_blank" class="keylink">vcD4NCjxoMiBpZD0="pms安裝路徑">PMS安裝路徑：

pms install 流程比較繁雜，只關注so相關的scanPackageDirtyLI函數中：

private PackageParser.Package scanPackageDirtyLI(PackageParser.Package pkg, int parseFlags,
            int scanFlags, long currentTime, UserHandle user) throws PackageManagerException {
            ...
             //invoke installer to do the actual installation  //作為外部apk 創建data目錄相關項
             //rameworks
ativecmdsinstalldcommands.c  install()中創建
                int ret = **createDataDirsLI**(pkgName, pkg.applicationInfo.uid,
                                           pkg.applicationInfo.seinfo);
            ...
    if (isSystemApp(pkg) && !isUpdatedSystemApp(pkg)) {
        ...

           setBundledAppAbisAndRoots(pkg, pkgSetting);

            ...

           setNativeLibraryPaths(pkg);  

    }
    else
    {
    setNativeLibraryPaths(pkg);
    ...
        if (isAsec) {
                        copyRet = NativeLibraryHelper.findSupportedAbi(handle, abiList);
                    } else {
                        copyRet = NativeLibraryHelper.copyNativeBinariesForSupportedAbi(handle,
                                nativeLibraryRoot, abiList, useIsaSpecificSubdirs);
                    }
    setNativeLibraryPaths(pkg);
      if (DEBUG_INSTALL) Slog.i(TAG, Linking native library dir for  + path);
            final int[] userIds = sUserManager.getUserIds();
            synchronized (mInstallLock) {
                // Create a native library symlink only if we have native libraries
                // and if the native libraries are 32 bit libraries. We do not provide
                // this symlink for 64 bit libraries.
                if (pkg.applicationInfo.primaryCpuAbi != null &&
                        **!VMRuntime.is64BitAbi(pkg.applicationInfo.primaryCpuAbi)**) {
                    final String nativeLibPath = pkg.applicationInfo.nativeLibraryDir;
                    for (int userId : userIds) {
if (mInstaller.linkNativeLibraryDirectory(pkg.packageName, nativeLibPath, userId) < 0)    {
                            throw new PackageManagerException(INSTALL_FAILED_INTERNAL_ERROR,
                                    Failed linking native library dir (user= + userId + ));
                        }
                    }
                }
            }

    }
}

看下system app 的安裝配置函數 setBundledAppAbisAndRoots：

    /**
     * Calculate the abis and roots for a bundled app. These can uniquely
     * be determined from the contents of the system partition, i.e whether
     * it contains 64 or 32 bit shared libraries etc. We do not validate any
     * of this information, and instead assume that the system was built
     * sensibly.
     */
    private void setBundledAppAbisAndRoots(PackageParser.Package pkg,
                                           PackageSetting pkgSetting) {
        final String apkName = deriveCodePathName(pkg.applicationInfo.getCodePath());

        // If /system/lib64/apkname exists, assume that is the per-package
        // native library directory to use; otherwise use /system/lib/apkname.
        final String apkRoot = calculateBundledApkRoot(pkg.applicationInfo.sourceDir);
        setBundledAppAbi(pkg, apkRoot, apkName);
        // pkgSetting might be null during rescan following uninstall of updates
        // to a bundled app, so accommodate that possibility.  The settings in
        // that case will be established later from the parsed package.
        //
        // If the settings aren't null, sync them up with what we've just derived.
        // note that apkRoot isn't stored in the package settings.
        if (pkgSetting != null) {
            pkgSetting.primaryCpuAbiString = pkg.applicationInfo.primaryCpuAbi;
            pkgSetting.secondaryCpuAbiString = pkg.applicationInfo.secondaryCpuAbi;
        }
    }

主要是在setBundledAppAbi中：

    /**
     * Deduces the ABI of a bundled app and sets the relevant fields on the
     * parsed pkg object.
     *
     * @param apkRoot the root of the installed apk, something like {@code /system} or {@code /oem}
     *        under which system libraries are installed.
     * @param apkName the name of the installed package.
     */
    private static void setBundledAppAbi(PackageParser.Package pkg, String apkRoot, String apkName) {
        final File codeFile = new File(pkg.codePath);
...
        if (has64BitLibs && !has32BitLibs) {
            // The package has 64 bit libs, but not 32 bit libs. Its primary
            // ABI should be 64 bit. We can safely assume here that the bundled
            // native libraries correspond to the most preferred ABI in the list.

            pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_64_BIT_ABIS[0];
            pkg.applicationInfo.secondaryCpuAbi = null;
        } else if (has32BitLibs && !has64BitLibs) {
            // The package has 32 bit libs but not 64 bit libs. Its primary
            // ABI should be 32 bit.

            pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_32_BIT_ABIS[0];
            pkg.applicationInfo.secondaryCpuAbi = null;
        } else if (has32BitLibs && has64BitLibs) {
            // The application has both 64 and 32 bit bundled libraries. We check
            // here that the app declares multiArch support, and warn if it doesn't.
            //
            // We will be lenient here and record both ABIs. The primary will be the
            // ABI that's higher on the list, i.e, a device that's configured to prefer
            // 64 bit apps will see a 64 bit primary ABI,

            if ((pkg.applicationInfo.flags & ApplicationInfo.FLAG_MULTIARCH) == 0) {
                Slog.e(TAG, Package:  + pkg +  has multiple bundled libs, but is not multiarch.);
            }

            if (VMRuntime.is64BitInstructionSet(getPreferredInstructionSet())) {
                pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_64_BIT_ABIS[0];
                pkg.applicationInfo.secondaryCpuAbi = Build.SUPPORTED_32_BIT_ABIS[0];
            } else {
                pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_32_BIT_ABIS[0];
                pkg.applicationInfo.secondaryCpuAbi = Build.SUPPORTED_64_BIT_ABIS[0];
            }
        } else {
            pkg.applicationInfo.primaryCpuAbi = null;
            pkg.applicationInfo.secondaryCpuAbi = null;
        }
 }

根據file 查找 確定primaryCpuAbi secondaryCpuAbi 變量值，這個也就決定了 這個 apk 由64bit 還是32bit 的zygote去fork 還有nativelibrary 查找的path

其中nativelibrary的幾個主要函數 setNativeLibraryPaths：

/**
     * Derive and set the location of native libraries for the given package,
     * which varies depending on where and how the package was installed.
     */
    private void setNativeLibraryPaths(PackageParser.Package pkg) {
        final ApplicationInfo info = pkg.applicationInfo;
        final String codePath = pkg.codePath;
        final File codeFile = new File(codePath);
        final boolean bundledApp = isSystemApp(info) && !isUpdatedSystemApp(info);
        final boolean asecApp = isForwardLocked(info) || isExternal(info);

        info.nativeLibraryRootDir = null;
        info.nativeLibraryRootRequiresIsa = false;
        info.nativeLibraryDir = null;
        info.secondaryNativeLibraryDir = null;

        if (isApkFile(codeFile)) {
            // Monolithic install
            if (bundledApp) {
                // If /system/lib64/apkname exists, assume that is the per-package
                // native library directory to use; otherwise use /system/lib/apkname.
                final String apkRoot = calculateBundledApkRoot(info.sourceDir);
                final boolean is64Bit = VMRuntime.is64BitInstructionSet(
                        getPrimaryInstructionSet(info));

                // This is a bundled system app so choose the path based on the ABI.
                // if it's a 64 bit abi, use lib64 otherwise use lib32. Note that this
                // is just the default path.
                final String apkName = deriveCodePathName(codePath);
                final String libDir = is64Bit ? LIB64_DIR_NAME : LIB_DIR_NAME;
                info.nativeLibraryRootDir = Environment.buildPath(new File(apkRoot), libDir,
                        apkName).getAbsolutePath();

                if (info.secondaryCpuAbi != null) {
                    final String secondaryLibDir = is64Bit ? LIB_DIR_NAME : LIB64_DIR_NAME;
                    info.secondaryNativeLibraryDir = Environment.buildPath(new File(apkRoot),
                            secondaryLibDir, apkName).getAbsolutePath();
                }
            } else if (asecApp) {
                info.nativeLibraryRootDir = new File(codeFile.getParentFile(), LIB_DIR_NAME)
                        .getAbsolutePath();
            } else {
                final String apkName = deriveCodePathName(codePath);
                info.nativeLibraryRootDir = new File(mAppLib32InstallDir, apkName)
                        .getAbsolutePath();
            }

            info.nativeLibraryRootRequiresIsa = false;
            info.nativeLibraryDir = info.nativeLibraryRootDir;
        } else {
            // Cluster install
            info.nativeLibraryRootDir = new File(codeFile, LIB_DIR_NAME).getAbsolutePath();
            info.nativeLibraryRootRequiresIsa = true;

            info.nativeLibraryDir = new File(info.nativeLibraryRootDir,
                    getPrimaryInstructionSet(info)).getAbsolutePath();

            if (info.secondaryCpuAbi != null) {
                info.secondaryNativeLibraryDir = new File(info.nativeLibraryRootDir,
                        VMRuntime.getInstructionSet(info.secondaryCpuAbi)).getAbsolutePath();
            }
        }
    }

根據pkg application info 來確定nativelibrarydir 依賴info中的 info.primaryCpuAbi

    private static String getPrimaryInstructionSet(ApplicationInfo info) {
        if (info.primaryCpuAbi == null) {
            return getPreferredInstructionSet();
        }

        return VMRuntime.getInstructionSet(info.primaryCpuAbi);
    }

非system apk 會調用 NativeLibraryHelper 的 findSupportedAbi 去解析 .apk 文件，根據系統suportabilist 去查找 lib目錄下的打包子目錄 找到匹配的abi

向文章開頭的那個 lib目錄 ，在64bit 機器上suportabilist為：

 public static final String[] SUPPORTED_ABIS = getStringList(ro.product.cpu.abilist, ,);

root@:/ # getprop ro.product.cpu.abilist                                 
arm64-v8a,armeabi-v7a,armeabi

會匹配arm64-v8a 賦值給 info.primaryCpuAbi

copyNativeBinariesForSupportedAbi 會去copy 前面匹配的lib 目錄到本地

最後設置NativeLibraryPaths ，
如果匹配的是64bit的，也就是arm64-v8a 那麼就不為/data/data/../lib 建立軟鏈接，這是與32bit 不同的地方

system.loadlibrary

作為動態庫加載的標准接口，直接看實現：

    public static void loadLibrary(String libName) {
        Runtime.getRuntime().loadLibrary(libName, VMStack.getCallingClassLoader());
    }

到Runtime.java中：

/*
     * Searches for and loads the given shared library using the given ClassLoader.
     */
    void loadLibrary(String libraryName, ClassLoader loader) {
        if (loader != null) {
            String filename = loader.findLibrary(libraryName);
            if (filename == null) {
                // It's not necessarily true that the ClassLoader used
                // System.mapLibraryName, but the default setup does, and it's
                // misleading to say we didn't find libMyLibrary.so when we
                // actually searched for liblibMyLibrary.so.so.
                throw new UnsatisfiedLinkError(loader +  couldn't find  +
                                               System.mapLibraryName(libraryName) + );
            }
            String error = doLoad(filename, loader);
            if (error != null) {
                throw new UnsatisfiedLinkError(error);
            }
            return;
        }

        String filename = System.mapLibraryName(libraryName);
        List candidates = new ArrayList();
        String lastError = null;
        for (String directory : mLibPaths) {
            String candidate = directory + filename;
            candidates.add(candidate);

            if (IoUtils.canOpenReadOnly(candidate)) {
                String error = doLoad(candidate, loader);
                if (error == null) {
                    return; // We successfully loaded the library. Job done.
                }
                lastError = error;
            }
        }

        if (lastError != null) {
            throw new UnsatisfiedLinkError(lastError);
        }
        throw new UnsatisfiedLinkError(Library  + libraryName +  not found; tried  + candidates);
    }

這裡的 ClassLoader loader 實際上會在 apk啟動的時候 初始化好一些相關的 子類 父類 還有參數

大體記錄一下 啟動時 初始流程 ：

ActivityThread.java   - handleBindApplication  
 final ContextImpl appContext = ContextImpl.createAppContext(this, data.info);
  LoadedApk pi = getPackageInfo(instrApp, data.compatInfo,
                    appContext.getClassLoader(), false, true, false);


ContextImpl.java  -getClassLoader()
LoadedApk.java    -getClassLoader()     ：  mLibDir = aInfo.nativeLibraryDir;
mClassLoader = ApplicationLoaders.getDefault().getClassLoader(zip, lib,
                        mBaseClassLoader);
ApplicationLoaders.java    -getClassLoader(...)
 PathClassLoader pathClassloader = new PathClassLoader(zip, libPath, parent);   //這裡的libPath 就是上面傳下來的aInfo.nativeLibraryDir

public class PathClassLoader extends BaseDexClassLoader
public class BaseDexClassLoader extends ClassLoader

loader.findLibrary(libraryName);

    /**
     * Finds the named native code library on any of the library
     * directories pointed at by this instance. This will find the
     * one in the earliest listed directory, ignoring any that are not
     * readable regular files.
     *
     * @return the complete path to the library or {@code null} if no
     * library was found
     */
    public String findLibrary(String libraryName) {
        String fileName = System.mapLibraryName(libraryName);
        for (File directory : nativeLibraryDirectories) {
            String path = new File(directory, fileName).getPath();
            if (IoUtils.canOpenReadOnly(path)) {
                return path;
            }
        }
        return null;
    }

這裡的nativeLibraryDirectories 即為前面一系列 構造時 設置了值 其中就有 aInfo.nativeLibraryDir

後面的邏輯就不去敘述了， 根據名字在這個目錄下去找 ，然後調用到本地JNI 最終調用 dlopen 加載打開so，必須是相同位數， 而這個關乎當前進程是屬於64bit 還是 32bit，這個會在zygote fork時區分， 同樣也是由PMS解析時得到的 info.primaryCpuAbi

AMS 請求zygote fork app process選擇

只關心 相關代碼 startProcessLocked函數：

    private final void startProcessLocked(ProcessRecord app, String hostingType,
            String hostingNameStr, String abiOverride, String entryPoint, String[] entryPointArgs) {
...

 String requiredAbi = (abiOverride != null) ? abiOverride : app.info.primaryCpuAbi;
            if (requiredAbi == null) {
                requiredAbi = Build.SUPPORTED_ABIS[0];
            }

            String instructionSet = null;
            if (app.info.primaryCpuAbi != null) {
                instructionSet = VMRuntime.getInstructionSet(app.info.primaryCpuAbi);
            }

            app.gids = gids;
            app.requiredAbi = requiredAbi;
            app.instructionSet = instructionSet;

            // Start the process.  It will either succeed and return a result containing
            // the PID of the new process, or else throw a RuntimeException.
            boolean isActivityProcess = (entryPoint == null);
            if (entryPoint == null) entryPoint = android.app.ActivityThread;
            checkTime(startTime, startProcess: asking zygote to start proc);
            Process.ProcessStartResult startResult = Process.start(entryPoint,
                    app.processName, uid, uid, gids, debugFlags, mountExternal,
                    app.info.targetSdkVersion, app.info.seinfo, requiredAbi, instructionSet,
                    app.info.dataDir, entryPointArgs);
            checkTime(startTime, startProcess: returned from zygote!);

...
}

Process中真正的socket 請求實現：

return zygoteSendArgsAndGetResult(openZygoteSocketIfNeeded(abi), argsForZygote);

openZygoteSocketIfNeeded 會根據傳下來的abi 去選擇 通信的socket

而在64bit 機器上，啟動時會 啟動 兩個 zygote service ，用於接收 64 32 的apk 請求：

service zygote /system/bin/app_process64 -Xzygote /system/bin --zygote --start-system-server --socket-name=zygote
    class main
    socket zygote stream 660 root system
    onrestart write /sys/android_power/request_state wake
    onrestart write /sys/power/state on
    onrestart restart media
    onrestart restart netd

service zygote_secondary /system/bin/app_process32 -Xzygote /system/bin --zygote --socket-name=zygote_secondary
    class main
    socket zygote_secondary stream 660 root system
    onrestart restart zygote

可以看到兩個 zygote 進程，基本一致 ，區別在於 64bit 32bit ，注冊socket不同
關於這兩個 zygote 進程啟動時的socket 注冊 就不多說了