Commons:Requests for comment/Technical needs survey/File upload stability

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search
Previous proposal Overview page Next proposal

File upload stability

[edit]

Description of the Problem

[edit]
  • Problem description: When uploading files using the UploadWizard or the API users experience very frequent problems resulting in aborted uploads or broken files. When the error is not recognized broken files or file description page info might be lost for Commons. If they are recognized they are very inconvenient to the uploads resulting in long term term contributors leaving or scaring new contributors.
  • Proposal type: bugfix
  • Proposed solution: Define the goal that only 1:10000 uploads using the API should fail because of server side problems. Only 1:1000 uploads should fail when uploading in the web browser because of server or website errors.

Discussion

[edit]
  • Diesem Vorschlag schließe ich mich aus tiefstem Herzen an. Insbesondere der UploadWizard könnte die Server-Fehlermeldungen viel verständlicher darstellen und viele auch besser abfangen. Ich möchte auch nochmals auf das Android-Tool Offroader hinweisen, das zeigt, wie stabil Uploads auf Commons mit der vorhandenen Server-Implementierung selbst unter widrigsten Bedingungen sein können, dass ein abgebrochener Upload ohne weiteres - auch auf einem anderen Gerät und mit einem anderen Internetzugang fortgesetzt werden kann, dass Uploads auf Fehlerfreiheit verifiziert werden können, dass Duplikate bereits vor Beginn eines Uploads erkannt und verhindert werden können und das - als Hilfe fürs Entwickeln, die Server-Meldungen während eines Uploads mitschneiden kann für ein PostMortem. --C.Suthorn (@Life_is@no-pony.farm - p7.ee/p) (talk) 18:41, 9 December 2023 (UTC)[reply]
I agree with this suggestion from the bottom of my heart. The UploadWizard in particular could display the server error messages much more comprehensibly and intercept many of them better. I would also like to point out again the Android tool Offroader, which shows how stable uploads to Commons can be with the existing server implementation, even under the most adverse conditions, that a canceled upload can easily happen - even on a different device and with a different Internet access can be continued, that uploads can be verified to be free of errors, that duplicates can be detected and prevented before an upload begins and that - as an aid to development, the server messages can be recorded during an upload for a postmortem.
translator: Google Translate via   — 🇺🇦Jeff G. please ping or talk to me🇺🇦 00:27, 10 December 2023 (UTC)[reply]
Yes, this is sorely needed! I like the idea of having target metrics especially. Nosferattus (talk) 17:40, 21 December 2023 (UTC)[reply]

I tried to gather some stats (These might not be fully robust, take with a bit of salt)

EventFeb 23Feb 24Feb 25Feb 26Feb 27Feb 28
[new code deployed]
Feb 29
Chunked upload recombine success11,16410,79611,16110,57510,42011,76610,457
Recieved last chunk11,19110,83911,21110,60910,47011,83010,495
Recieved first chunk11,23110,88011,28410,69110,50411,89210,555
Stash error (excluding logged out, may include duplicates from multiple attempts)86930613827677508232

The stash error row might be misleading, as most of them are all for the same user and might represent a bot with a programming mistake. Bawolff (talk)

Votes

[edit]
  1. yes.--RZuo (talk) 21:11, 21 January 2024 (UTC)[reply]
  2.  Support.   — 🇺🇦Jeff G. please ping or talk to me🇺🇦 21:26, 21 January 2024 (UTC)[reply]
  3. qualified yes. Of course I would welcome improvements in this area and think it would be a good use of resources, but I think the target metric of 1 server-side failure in 10,000 uploads is probably too ambitions. - Jmabel ! talk 23:47, 21 January 2024 (UTC)[reply]
  4.  Support — Draceane talkcontrib. 08:10, 23 January 2024 (UTC)[reply]
  5.  Support Jklamo (talk) 08:37, 23 January 2024 (UTC)[reply]
  6.  Support I thought only I had this issue. It happens to me every few uploads that I have to remove (some) files because they don't load.Paradise Chronicle (talk) 16:50, 24 January 2024 (UTC)[reply]
  7.  Support --Fl.schmitt (talk) 06:42, 26 January 2024 (UTC)[reply]
  8.  Support Daniel Case (talk) 02:37, 2 February 2024 (UTC)[reply]
  9.  Support Chunked upload code should be torn out and rewritten. Bawolff (talk) 18:46, 3 February 2024 (UTC)[reply]
  10.  Support Pleeeeaaase! I also have problems uploading files above 2 GiB right now... --PantheraLeo1359531 😺 (talk) 18:50, 3 February 2024 (UTC)[reply]
    @PantheraLeo1359531: Please see phab:T355433.   — 🇺🇦Jeff G. please ping or talk to me🇺🇦 10:59, 8 February 2024 (UTC)[reply]
    Ah, thanks! --PantheraLeo1359531 😺 (talk) 14:50, 9 February 2024 (UTC)[reply]
  11.  Support Yesterday, I encountered this issue where I had to upload this image again. It can be found in Image history --iMahesh (talk) 10:29, 8 February 2024 (UTC)[reply]
  12.  Support --ToprakM 15:04, 12 February 2024 (UTC)[reply]
  13.  Support Jo-Jo Eumerus (talk) 12:13, 13 February 2024 (UTC)[reply]
  14.  Support as proposer --GPSLeo (talk) 18:28, 13 February 2024 (UTC)[reply]