Add std::fs::{Home|Media}Dirs

[CAD97]

View all comments

• ACP: ACP: std::fs::UserDirs or similar libs-team#830

Replacement for std::os::unix::xdg as suggested by libs-api in #157515 (comment). Exposes media directories common between the three big OSes in addition to the cache/config/data/state directories under a separate feature gate. API summary:

// mod std::fs
pub struct HomeDirs { /* ... */ }
impl HomeDirs {
    fn empty() -> Self;

    pub fn config_home(&self) -> Option<&Path>;
    pub fn data_home(&self) -> Option<&Path>;
    pub fn state_home(&self) -> Option<&Path>;
    pub fn cache_home(&self) -> Option<&Path>;

    pub fn set_config_home(&mut self, path: PathBuf) -> &mut Self;
    pub fn set_data_home(&mut self, path: PathBuf) -> &mut Self;
    pub fn set_state_home(&mut self, path: PathBuf) -> &mut Self;
    pub fn set_cache_home(&mut self, path: PathBuf) -> &mut Self;
}

pub struct MediaDirs { /* ... */ }
impl MediaDirs {
    pub fn empty() -> Self;

    pub fn desktop(&self) -> Option<&Path>;
    pub fn documents(&self) -> Option<&Path>;
    pub fn downloads(&self) -> Option<&Path>;
    pub fn music(&self) -> Option<&Path>;
    pub fn pictures(&self) -> Option<&Path>;
    pub fn videos(&self) -> Option<&Path>;

    pub fn set_desktop(&mut self, path: PathBuf) -> &mut Self;
    pub fn set_documents(&mut self, path: PathBuf) -> &mut Self;
    pub fn set_downloads(&mut self, path: PathBuf) -> &mut Self;
    pub fn set_music(&mut self, path: PathBuf) -> &mut Self;
    pub fn set_pictures(&mut self, path: PathBuf) -> &mut Self;
    pub fn set_videos(&mut self, path: PathBuf) -> &mut Self;
}

// mod std::os::darwin::fs
impl HomeDirsExt for HomeDirs { /* ... */ }
pub impl(self) trait HomeDirsExt {
    fn sysdir() -> io::Result<Self>;
}

impl MediaDirsExt for MediaDirs { /* ... */ }
pub impl(self) trait MediaDirsExt {
    fn sysdir() -> io::Result<Self>;
}

// mod std::os::unix::fs
impl HomeDirsExt for HomeDirs { /* ... */ }
pub impl(self) trait HomeDirsExt {
    fn xdg() -> io::Result<Self>;

    fn runtime_home(&self) -> Option<&Path>;
    fn config_dirs(&self) -> Option<XdgDirs<'_>>;
    fn data_dirs(&self) -> Option<XdgDirs<'_>>;

    fn set_runtime_home(&mut self, path: PathBuf) -> &mut Self;
    fn set_config_dirs(&mut self, paths: OsString) -> &mut Self;
    fn set_data_dirs(&mut self, paths: OsString) -> &mut Self;
}

impl MediaDirsExt for MediaDirs { /* ... */ }
pub impl(self) trait MediaDirsExt {
    fn xdg() -> io::Result<Self>;

    fn templates(&self) -> Option<&Path>;
    fn set_templates(&mut self, path: PathBuf) -> &mut Self;
}

pub struct XdgDirs<'a> { /* ... */ }
impl Iterator for XdgDirs<'a> {
    type Item = &'a Path;
    /* ... */
}

// mod std::os::windows::fs
impl HomeDirsExt for HomeDirs { /* ... */ }
pub impl(self) trait HomeDirsExt {
    fn appdata_env() -> io::Result<Self>;
    fn known_folders() -> io::Result<Self>;
}

impl MediaDirsExt for MediaDirs { /* ... */ }
pub impl(self) trait MediaDirsExt {
    fn known_folders() -> io::Result<Self>;
}

Potentially outdated info

---

This implementation diverges from the directories crate's mapping in that we set state_dir in the non-unix constructors (to ~/Library/Application Support on Darwin and %APPDATA% on Windows). This mapping is derived from the idea that "state" files are application support files that are not important nor portable enough to the user to be "data" files.

---

The XDG paths are as described in the XDG Base Directories Specification and the xdg-user-dirs tool. $XDG_CONFIG_DIR/user-dirs.dirs is parsed directly to avoid delegating to potentially arbitrary shell execution.

The Darwin paths are loaded via the sysdir(3) API from libSystem.dylib (introduced in macOS 10.12 with a similar timeline for other Darwin OSes, deprecating the earlier NSSystemDirectories.h API). Using the File System Effectively points to preferring the Foundation framework's NSSearchPathForDirectoriesInDomain(_:_:_:) or NSFileManager.URLForDirectory instead, but calling those correctly requires an active Objective C autorelease pool, IIUC. The Library/Application Support directory is used for config_home, data_home, and state_home; the Apple documentation The Library Directory Stores App-Specific Files directly calls out placing data and configuration files in Library/Application Support, and state files are just less user-meaningful data files.

The Windows paths are loaded via the Known Folders API (introduced in Vista). config_home and data_home are placed in AppData\Roaming as files intended to be important and portable to the user, while cache_home and state_home are placed in AppData\Local as files that aren't.

---

I'm not fully confident about the handling of the XDG base directory paths which don't have good cross-platform analogs, as well as the exact API for the search path dealing functions, but I'm confident that the shape of the rest of the API does match the stdlib API style. Common paths are platform-independent enough of a needed concept to be exposed by std, IMHO, but platform-specific that a struct with public fields (even #[non_exhaustive]) seems incorrect, specifically because of platform-specific paths that we may want to expose like is already done for XDG.

The one API change I could see doing is moving state_dir into the XDG UserDirsExt. I chose not to do this for this initial implementation, though, as getting the ideal choice of fallback for both Darwin and Windows can't be achieved in an OS-agnostic way:

-	Darwin	Windows
cache	~/Library/Caches	~/AppData/Local
config	~/Library/Application Support	~/AppData/Roaming
data	~/Library/Application Support	~/AppData/Roaming
state	~/Library/Application Support	~/AppData/Local

A more drastic change would be to move all four onto the unix UserDirsExt, adding caches/application_support to the Darwin UserDirsExt and roaming_app_data/local_app_data to the Windows UserDirsExt. This would be more "correct" but seems a bit heavy-handed, as it would mean applications need to pull in OS-specific extension traits just to place their support files in something more appropriate than a ~/.appname directory.

We could also separate the "home" directory API from the "media" directory API. I'm neutral on this with one relevant note: the app-specific cache/config/data/state files need a subdirectory named after the application, so "ProjectDirs" would exclude the media directories; it could make sense to have a type with just those and a push_application_subdir method.

Switching the impl to using a pal imp::UserDirs could be reasonable, but seems at odds with the desire to have the target agnostic way to "build your own" UserDirs. An ExtraUserDirs instead of the #[allow(dead_code)] fields would make sense, I just didn't know how to best set up that in the pal layer.

---

Disclaimer: This was worked on as part of my employment at Canonical. I initially proposed it independently of my employment, but improving std's functionality is part of my job description, so Canonical told me I should use work time on it.

AI Disclosure: I did not use AI to generate any of the code, with a partial exception for VSCode's AI-assisted smart autocomplete helping with the repetitive parts of the code. All nontrivial code was handwritten. As an experiment, I did use some AI to assist in exploring the problem and API design spaces.

I tested locally on my Ubuntu developer machine, but am relying on CI for Darwin and Windows tests. 🤞

---

cc @joshtriplett @nia-e

[madsmtm]

Reviewed the Darwin parts.

calling those correctly requires an active Objective C autorelease pool, IIUC

Not that hard though, you can push and pop it with objc_autoreleasePoolPush/objc_autoreleasePoolPop. The bigger problem is that it requires linking Foundation, which has a startup cost we'd rather avoid.

This implementation diverges from the directories crate's mapping [...]

It seems to me that for something as nuanced as these user dirs (with a lot of platform-specific details that are not readily apparent), it might make sense to implement the desires std API in directories first? And once it stabilizes more there, we could upstream it to std?

View changes since this review

[CAD97]

It seems to me that for something as nuanced as these user dirs (with a lot of platform-specific details that are not readily apparent), it might make sense to implement the desires std API in directories first?

This is mostly already the case. The only API-facing changes from directories here are:

• Trimming the less cleanly portable API surface
• Returning Option everywhere (requested by T-libs-api)
• Setter methods for custom setups (requested by T-libs-api)
• Explicit constructors for each convention (requested by T-libs-api)
• Making state_dir fall back to what directories calls data_local_dir for non-XDG constructors
• Combining UserDirs/BaseDirs (probably going to revert this)

The ideal API shape inside std and in a crate often differ slightly. This approved impl experiment is to determine if a form of this API that fits std's goals exists.

---

I'm going to split the base directory discovery and the user/media directories into different types to better represent that the existence of these sets is not strongly correlated and fix the things @madsmtm pointed out w.r.t. docs and the darwin impl, then this should be good for proper libs-api review.

The use of shlex for shell-unquote for the XDG user dirs needs a resolution, but doing the work to give shlex a rustc-dep-of-std feature can wait until we know whether that's the direction we want to take.

[madsmtm]

r=me on the Darwin impl now (haven't tested it this time around, but pretty sure CI will catch it if it doesn't work), thanks!

Don't have much of an opinion on the API design and the other platforms, I'll leave that to someone else.

View changes since this review

[CAD97]

I consider this fully ready for review now.

r? @rust-lang/libs-api

[CAD97]

@rustbot author

Rebasing and applying requested updates from ACP.

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[rustbot]

⚠️ Warning ⚠️

• There are issue links (such as #123) in the commit messages of the following commits.
  Please move them to the PR description, to avoid spamming the issues with references to the commit, and so this bot can automatically canonicalize them to avoid issues with subtree.
  • f290c98

[CAD97]

@rustbot ready

Commits nicely split the units of work again :3

[rust-bors]

☔ The latest upstream changes (presumably #162622) made this pull request unmergeable. Please resolve the merge conflicts by rebasing.