8,012
edits
Difference between revisions of "Cloud Storage Support"
Jump to navigation
Jump to search
adding info about the 2016 GSoC work
(Created page with "== Overview == There are two main features of the Cloud: sync saves between the device and cloud storage and download game data from cloud storage. Syncing is automatic, but...") |
(adding info about the 2016 GSoC work) |
||
| (4 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
Work was done by the GSoC student [[User:Tkachov|Tkachov]] during the [[Summer of Code/GSoC2016|GSoC 2016]] to implement cloud storage support in ScummVM. | |||
== Overview == | == Overview == | ||
| Line 4: | Line 6: | ||
== Connecting a Storage == | == Connecting a Storage == | ||
[[File:Cloud_tab.png|thumb|200px|Cloud tab]] | |||
A new Cloud tab is added to Options dialog, and that's where user could choose the active Storage. It's "<none>" by default, and the following Cloud providers are supported to the moment: Dropbox, OneDrive, Google Drive and Box ([[Cloud_Storages_API_comparison|API analysis/comparison]]). | A new Cloud tab is added to Options dialog, and that's where user could choose the active Storage. It's "<none>" by default, and the following Cloud providers are supported to the moment: Dropbox, OneDrive, Google Drive and Box ([[Cloud_Storages_API_comparison|API analysis/comparison]]). | ||
When selecting a non-connected Cloud provider, only "Connect" button is available. It opens a Storage Connection Dialog, where user sees the instructions on how to connect a Storage. | When selecting a non-connected Cloud provider, only "Connect" button is available. It opens a Storage Connection Dialog, where user sees the instructions on how to connect a Storage. | ||
[[File:Storage_Connection_Dialog_8_fields.png|thumb|200px|Storage Connection Dialog]] | |||
Basically, ScummVM uses REST APIs of the specified Cloud providers. In order to do that, a special application has to be registered in the developers section of these providers. Each application has a KEY and a SECRET strings. Developers can also specify some information about their applications (name, logo, description) and add permitted redirect_uris. | Basically, ScummVM uses REST APIs of the specified Cloud providers. In order to do that, a special application has to be registered in the developers section of these providers. Each application has a KEY and a SECRET strings. Developers can also specify some information about their applications (name, logo, description) and add permitted redirect_uris. | ||
| Line 19: | Line 25: | ||
So, it's obvious that KEY is not a secret, and even called "id" in some providers. We also specify a redirect_uri there, which is required to be either <code>https://</code> link, or a <code>http://localhost[:port]</code> one. Response type "code" means that we want to get a code, not the actual access_token. | So, it's obvious that KEY is not a secret, and even called "id" in some providers. We also specify a redirect_uri there, which is required to be either <code>https://</code> link, or a <code>http://localhost[:port]</code> one. Response type "code" means that we want to get a code, not the actual access_token. | ||
When users open such link, they have to auth to the provider first. When they do, they'd see some information about the application and would be given a choice to allow or deny that application access to user's storage. There different "scopes", which could help application to specify what they want to get access to (for example, application could be limited to read/write files only in a special directory). Providers usually show which scopes application wants to use, so we shouldn't ask for everything if we need only a few permissions. | When users open such link, they have to auth to the provider first. When they do, they'd see some information about the application and would be given a choice to allow or deny that application access to user's storage. There different "scopes", which could help application to specify what they want to get access to (for example, application could be limited to read/write files only in a special directory). Providers usually show which scopes application wants to use, so we shouldn't ask for everything if we need only a few permissions ([[Cloud_Storage_Scopes_List|list of scopes used]]). | ||
[[File:Code_Page.png|thumb|200px|https://www.scummvm.org/c/code page]] | |||
If users allow the application, provider redirects them to the redirect_uri with code passed as GET parameter. This code should be somehow passed to ScummVM, so it could request the access_token needed to use the API. | If users allow the application, provider redirects them to the redirect_uri with code passed as GET parameter. This code should be somehow passed to ScummVM, so it could request the access_token needed to use the API. | ||
[[File:Storage_Connection_Dialog_no_fields.png|thumb|200px|Storage Connection Dialog (no fields)]] | |||
ScummVM supports two different ways to get that code. The first is using <code>https://www.scummvm.org/c/code</code> page, where the long code is automatically transformed into a few short groups with hashsum added. The Storage Connection Dialog then contains 8 fields, where these short groups should be typed in. It automatically checks the hashsums, so if there is a typo, ScummVM would notify user about it. If no typos are detected, "Connect" button would be enabled. When user presses it, hashsum characters are removed and the passed code used to receive access_token. | ScummVM supports two different ways to get that code. The first is using <code>https://www.scummvm.org/c/code</code> page, where the long code is automatically transformed into a few short groups with hashsum added. The Storage Connection Dialog then contains 8 fields, where these short groups should be typed in. It automatically checks the hashsums, so if there is a typo, ScummVM would notify user about it. If no typos are detected, "Connect" button would be enabled. When user presses it, hashsum characters are removed and the passed code used to receive access_token. | ||
| Line 50: | Line 60: | ||
== Using Storage API == | == Using Storage API == | ||
[[File:Cloud_Icon.png|thumb|200px|Cloud icon]] | |||
All Cloud-related methods require making a HTTP request to the provider's REST API (and even more than one request sometimes). As that's not a synchronous operation, all such methods return a Request *, which could be used to control the request. To get the result, one must pass callbacks (one for success and one for failure) to the method, and Request would call one on these when it's complete. More information on Requests system is available on a [[Cloud_backends_sequence_diagrams|separate page]]. | All Cloud-related methods require making a HTTP request to the provider's REST API (and even more than one request sometimes). As that's not a synchronous operation, all such methods return a Request *, which could be used to control the request. To get the result, one must pass callbacks (one for success and one for failure) to the method, and Request would call one on these when it's complete. More information on Requests system is available on a [[Cloud_backends_sequence_diagrams|separate page]]. | ||
| Line 60: | Line 72: | ||
== Downloading game data == | == Downloading game data == | ||
[[File:Download_Dialog.png|thumb|200px|Download dialog]] | |||
When Storage is connected, two new buttons appear in the Cloud tab: "Refresh" and "Downloads". The first one initiates user information refresh (such as email or used/available bytes) and the second open a special Download Dialog. | When Storage is connected, two new buttons appear in the Cloud tab: "Refresh" and "Downloads". The first one initiates user information refresh (such as email or used/available bytes) and the second open a special Download Dialog. | ||
| Line 69: | Line 83: | ||
== Syncing saves == | == Syncing saves == | ||
If Storage is connected, saves sync would be started on ScummVM launch, on each manual and autosave and when Save/Load Dialog is opened. If one saves sync is already running, the other would be scheduled after the first one is finished to check whether there are new files to sync. | If Storage is connected, saves sync would be started on ScummVM launch, on each manual and autosave, when a storage is connected and when Save/Load Dialog is opened. If one saves sync is already running, the other would be scheduled after the first one is finished to check whether there are new files to sync. | ||
Loading from files which are being synced is not available. A popup with progress bar is shown in the Save/Load Dialog while ScummVM downloads files from the remote storage. In games with "complex" saves names loading is completely unavailable while files are being downloaded. While ScummVM uploads files, it's safe to use these files already. | Loading from files which are being synced is not available. A popup with progress bar is shown in the Save/Load Dialog while ScummVM downloads files from the remote storage. In games with "complex" saves names loading is completely unavailable while files are being downloaded. While ScummVM uploads files, it's safe to use these files already. | ||
| Line 76: | Line 90: | ||
Saves sync actually syncs all the files within the "ScummVM/Saves" remote directory and "Saved games" local directory (the savespath, that is). Directories are considered flat, so subdirectories are not synced (if there are any). A special "timestamps" local file is used to determine which files should be synced. It contains the remote "modified at" timestamps for each file. If remote file has a newer timestamp or is not listed, it's being downloaded. If local file has a newer timestamp or is not listed (but exists), it's being uploaded. We download files first, upload second. Timestamps are updated after each downloaded and uploaded file, so if sync is interrupted, next sync will continue from the right state. | Saves sync actually syncs all the files within the "ScummVM/Saves" remote directory and "Saved games" local directory (the savespath, that is). Directories are considered flat, so subdirectories are not synced (if there are any). A special "timestamps" local file is used to determine which files should be synced. It contains the remote "modified at" timestamps for each file. If remote file has a newer timestamp or is not listed, it's being downloaded. If local file has a newer timestamp or is not listed (but exists), it's being uploaded. We download files first, upload second. Timestamps are updated after each downloaded and uploaded file, so if sync is interrupted, next sync will continue from the right state. | ||
When file fails to upload, storage providers ignore that there was some uploading session and file is not created or modified. ScummVM stops the sync, and tries to upload that file again on next sync. | |||
When file fails to download, ScummVM deletes the invalid file and stops the sync. As its timestamp is not updated, ScummVM would try to download the file again on next sync. | |||