From Traditional File APIs to SAF: How Esper Redefined Content Management in the DevOps World

At Esper, we don’t believe in limiting features, including storage. But everything has a starting point, and this is the story of how we conquered the complete device storage step by step, starting from zero, giving customers a captain’s seat to control complete device storage.

The Beginning 

In the initial phase, storage wasn’t a prioritized component. Eventually, we built our own Content Management System (CMS), giving customers additional power over saving device files. 

Initially, our CMS feature could only download and save files in Esper’s package or Downloads folder. With this in place, there wasn’t a dire reason to further improve the capabilities of our CMS — until a requirement came from a customer, an educational institution, who needed to save lecture files in removable storage on devices. It was at this moment we realized the limitations of our CMS feature. We noted three key issues:

• Our existing CMS feature couldn’t save files anywhere on an SD card 
• Unlike internal storage, each SD card has different IDs, so IT admins need to know the IDs of every device. In this case, if a student replaces an SD card, it would be a very tedious process to figure out the ID again
• If files are saved in our package directory in Android 10 and above, other apps can not access them

Despite all these limitations, we still managed to provide a solution for the above case within a day. Here’s how we did it.

What is SAF?

Ok, so let's start with Storage Access Framework (SAF). SAF is an Android system feature that enables apps to interact with files across various storage locations, including internal storage, public directories, and removable storage like SD cards. SAF provides a standardized way for apps to request and manage file access, offering a more controlled and secure method of handling files than traditional file access methods.

Why Didn’t We Use SAF In the Beginning?

Initially, we didn’t implement SAF due to its requirement for user consent. SAF needs users to manually grant permission to access specific directories, which isn’t feasible for apps like Device Policy Controllers (DPC) that require seamless and uninterrupted file management. The need for constant user interaction to grant permissions was not ideal for core management applications.

Why Do We Need SAF Now?

While traditional file storage solutions were sufficient in the past, challenges emerged with SD card access and compatibility with newer Android versions. SAF became essential for several reasons:

• Access to SD Cards: SAF enables DPC to store and manage files on removable storage such as SD cards, which are subject to increased restrictions in newer Android versions
• Compliance with Android 10+ Restrictions: As Android 10 and later versions introduced stricter file access rules, SAF offers a compliant way to manage files under these new constraints

By adopting SAF, our app can effectively handle internal and external storage, ensuring compatibility and compliance with the latest Android standards.

Handling Directory Permissions with SAF

Directory permissions are necessary to create or access files using SAF. Here's how we handle the process of requesting these permissions:

Receiving a Directory Path

When a SyncContent (CMS) command is sent to the device, it includes a specified path. For example:

/storage/emulated/0/Esper/NewFolder/

If the device receives this path and does not have the required permission, it immediately returns a failure message indicating the lack of permission.

Forcing User Permission

If the admin wants the user to grant permission upon receiving the command, you can prefix the path with the forced/ keyword. For instance:

forced/storage/emulated/0/Esper/NewFolder/

When the device receives a command with this path without sufficient permission, the user is taken directly to the specified directory to grant access.

Important Considerations

• Android Version Limitations: This direct navigation to the path works only on devices running Android 8 or above. For devices below Android 8, admins must instruct users to grant permission to the root volume itself, thereby obtaining access to all directories.
• Possible Failures:
  • The user may choose not to grant permission at all.
  • The user may grant permission to a different directory than the one requested.

In both scenarios, the command is marked as failed.

Key Features of Esper’s CMS with SAF

1. Automatic Folder Tree Creation

When sending a command to save a file in a specific directory (e.g., /a/b/c/d), and the DPC (Device Policy Controller) already has permission for a higher-level directory (e.g., /a/b), the DPC automatically creates the remaining subfolders (c/d).

• Smart Folder Creation: Before creating any subfolder, the system checks if a document with the same name exists and is not a directory, avoiding conflicts
• No Duplicate Folders: DPC ensures no duplicate folder creation, maintaining a clean and organized file structure

2. Precise File Creation with SAF

Creating files using the Storage Access Framework (SAF) requires both the file name and MIME type.

• Mandatory MIME Type: If a command lacks a MIME type, the DPC will not create the file, ensuring only properly formatted documents are generated.
• No Duplicate Files: DPC prevents the creation of duplicate files within a directory, reducing clutter and confusion.

3. Permission Management via Esper Settings

Granting storage permissions is made easy with the new Storage option in Esper Settings.

• User-Controlled Access: Users can grant permissions for internal and external removable storage (such as SD cards) directly through Esper Settings.

Dynamic External Storage Management: If removable storage is not present or mounted, the external storage option is automatically disabled, providing a clear, intuitive user experience.

4. Smart Path Suggestions with SAF

When a command fails due to a lack of required permissions, the DPC will automatically suggest alternative paths that have already been granted permission by the user, either through Esper Settings or from previous commands. This helps ensure smoother operations without unnecessary interruptions.

5. Unified Path Scheme for Flexible Storage Access

For scenarios where admins need to push files to a group of devices, whether on internal or external storage, SAF introduces a Unified Path Scheme. This scheme allows consistent file operations across devices, even when the exact storage path differs.

• Streamlined File Deployment: Each device may have different path identifiers (e.g., for external storage like SD cards: /storage/0ECD-3M11), but with the Unified Path Scheme, admins can specify a common format that applies universally, enabling efficient file deployment across multiple devices.

This feature ensures seamless management of files across diverse storage locations, whether internal or external, without needing device-specific adjustments.

Admins can use a scheme format to specify paths that work across devices, even if SD card IDs differ:

Note: External schemes require a mounted SD card and enabled External Device policy.

Conclusion

In conclusion, integrating SAF into Esper's content management system has revolutionized file handling in the DevOps world. By addressing the challenges posed by traditional file APIs and evolving Android restrictions, SAF ensures seamless access to internal and external storage while maintaining compliance and security. Key features such as automatic folder creation, precise file management, and user-controlled permissions through Esper Settings enhance the overall efficiency and user experience. Moreover, introducing smart path suggestions and a unified path scheme further streamlines file operations, making Esper's CMS a robust and adaptable solution for modern DevOps needs.