Troubleshooting iCloud Photo Sync on Windows – WARNING OBSOLETE

December 8th, 2020

UPDATE: I got 80% of the way through this blog-as-I-debug post before realizing that while iTunes happily pestered me into updates every time I opened it, iCloud was not upgraded along with it. I was on version 7, after upgrading to 11, things changed a lot. I’m publishing this unfinished post since some of it was interesting, and maybe someone who insists on staying on Version 7 of iCloud will have some use for it. There was some interesting SQLite browsing that I found zero references to in my searching. Possibly SQLite still backs iCloud, I haven’t found it though, also, I haven’t tried.

UPDATE 2: After writing the above, I decided to do a dir /s client.db and found a likely new location for the database in current versions of iCloud for Windows – C:\Users\[username]\AppData\Local\Packages\AppleInc.iCloud_nzyj5cx40ttqa\LocalCache\Local\Apple Inc\iCloudPhotoLibrary
I’m not sure if the random character section changes per install or per version or what.

——————

My iPhone upgrade seemed to go magically. That weird cloud pattern and sync was soooo smooth, that obviously it couldn’t be perfect. I noticed eventually that the photo count was wrong.

To be fair, my photo setup was kind of messed up, I’ve been moving files off manually and the photos have migrated over two phones previously. It was a good thing I made a manual backup of the “DCIM” folder.

I only noticed a few days afterwards, so I didn’t want to do a full restore from iTunes – this might not have been the best choice in the end.

After much fiddling, I’ve decided to try and use iCloud photos to get the photos back onto the phone. I hope that I will be able to use it to manage my photos over the long term better anyway, manually grabbing files off the DCIM folder isn’t ideal anymore. The Windows integration was and still is pathetic (which is why this blog post is being written); but, at least the web interface is much better now. I think.

Moving all the JPG only to C:\Users\[username]\Pictures\iCloud Photos\Uploads (the default folder for uploads from my Windows 10 PC).

I don’t have that many photos that I want to keep on my phone… only about 1800 but of course, it didn’t work.

And for some reason, I can’t find any good debugging information.

Check iCloud version, it’s in fine print towards top right when you have iCloud program window open. I started with 7.21, newest is 11 – and it did NOT auto-update. (i may have disabled an auto-update feature in the past)

iCloud Photos is stuck with 300 images left to upload and synchronize. And no indication of what is stuck or why. And multiple web searches turned up only partial help.

First, the work database used by the iCloud Photos sync is in this folder: C:\Users\[username]\AppData\Local\Apple Inc\iCloudPhotoLibrary\client.db

With a little wizardry figured out it was a SQLite database.
(I just opened the file in a text editor and looked for hints in the text – it’s was pretty obvious looking at the head of the file.)

I’m lazy, so I found a GUI: https://sqlitebrowser.org/

I cracked open the client.db file, and I’m more than happy to see that the Apple writers made the clearest design that I could have hoped for.

I found the list of files to be uploaded in the table “files_to_upload” (WOW).

As far as I can tell so far, there is a record for every file that has ever been in the Photos\Uploads folder, and also any files that were manually added via the Tray Icon.

Sort the table by the upload_state column. I only have 2 values in my table:

  • 0 seems to be not uploaded – it matches my stuck file count exactly
  • 2 seems to be upload done

Other columns:

  • path – The file in question is in the “path”
  • cpl_asset_record_id – none of the files that did not upload have this blank; however, a few of the ones that are uploaded also have a blank here – I suspect this is something akin to “cloud” ID
  • mmcs_file_hash – I suspect it’s a hash of the file.
  • path_hash – this is set as the primary key, I would assume (maybe) uploads are done in path_hash ASC order?
  • ROWID – apparently SQLite has ROWID always that can be queried – maybe this is used for order of uploads

I thought maybe this would find some duplicates (because I know there are some) – but no… SELECT COUNT(*) Kount, mmcs_file_hash FROM files_to_upload GROUP BY mmcs_file_hash HAVING Kount > 1

I know that when it is stuck on a file, it does go on to the next one – my test batch of 20 or so files had a couple stuck, but when I uploaded the rest – most of the later ones uploaded.

Is it the file? I made a copy of one to “IMG_5893 – Copy.JPG”

PNG files (screenshots) are NOT handled (WTF). MOV movie files no too. I don’t have a plan, I might convert the PNG files to JPG and just be done with those. MOV I might move to Google Drive or One Drive, or leave the off the phone altogether.

Event Log? I don’t see any in common logs, nor do I see an Apple file.

iCloud executables seem to be here: C:\Program Files (x86)\Common Files\Apple\Internet Services – no signs of log files

Some logs here: C:\Users\lchan\AppData\Roaming\Apple Computer\Logs-Old (found this after upgrading to version 11, I suspect folder was named “Logs” before)

Argh. After all of the above, I found out I was running a really old version (7) of iCloud and iCloud Photos.

C:\Program Files\WindowsApps\AppleInc.iCloud_11.4.12.0_x86__nzyj5cx40ttqa\iCloud

TODO: I got tired of this task and gave up, but the next steps would be to take the list of files in the database that did not upload, write some sort of batch or PowerShell or Python script to copy those out to a new folder, and then try the upload again, I guess. Please comment below if you take it that far.

Installing Older Versions of Apps (e.g. NPR One) on Older Versions of iOS on a New Apple ID

December 7th, 2020

Hmm… quite a long gap between blog posts… again. It’s amazing how time flies.

Here is a quick workaround I figured out today on how to get an old version of an app installed on a new-to-you device that is running an older version of iOS than the minimum required by the current version of the app. (Run on sentence much?)

My specific situation: I just got an old iPhone 5S, it’s actually a hand-me-down-and-back. I wanted to run some streaming audio on it in the kitchen. Unfortunately, one app, NPR One, required a minimum iOS version of 13. The last major version of iOS for the iPhone 5S is version 12.

The Apple App Store will prompt you to install the last compatible version of an app (from App Store >> Account >> Purchases), but ONLY if you had installed it previously under your Apple ID.

I didn’t want my real Apple ID sitting around on a device with a “throwaway” PIN, so I had created a new Apple ID with a secondary (actually tertiary) email account I have. This brand new account did NOT have the NPR One “purchased”. So I was stuck. Or was I?

My workaround was to log into my primary Apple ID on the new-old phone, then go to the App Store and install NPR One from “Purchases”. The App Store happily prompted and allowed me to install whatever older version of NPR One is supported by iOS 12. This worked great.

Then I logged out of my primary Apple ID on the old phone, logged back in with the new disposable Apple ID, and I was happy to see that the application was still there!

I am crossing my fingers that this old version of NPR One will continue to stream for 2-3 years until I do another round of waterfalling device upgrades.

P.S. Weirdly, this ended up forking my Wallet. I had my Wallet items on my regular phone under my regular Apple ID, and a copy of Wallet (except credit card, I think), on my 5S phone under it’s new “not secure” Apple ID. I manually removed the Wallet items from the 5S. I guess this is a design feature, except, I didn’t want someone picking up this old phone from my kitchen and going wild spending my collected loyalty points!

Unrecognizable Credit Card Descriptions

November 27th, 2017

I’ve decided to start journal some undecipherable credit card descriptions. For my own future reference.

Maybe it will also help clear up some confusion for others. Feel free to comment with your own strange ones.

“1717 GB VANCOUVER MARI VANCOUVER” – It took me some guessing and testing (while I was on hold with my credit card company) to figure out that this was legitimate, it’s the Gas Bar at the Superstore complex on Marine Drive in Vancouver. I can’t be the only person who couldn’t recognize this charge at first!

“CPC SCP/0000320901359” – If you guessed Canada Post, you win! Seriously? How can you not get charge backs?

Update: Apparently I’m not the only one making a list. TD has one here.

 

fping (Windows) – Download the last version.

November 10th, 2017

Hmm… two posts in a row about stuff I’ve used for years that have now gone offline. There’s a statement here about the so called longevity of things in the digital era. Hey, looking at the calendar widget at the right of this blog, it just hit 10 years of age!

fping, apparently short for “Fast Pinger”, is a really handy tool I use when I want a simple way to monitor an IP address over a period of hours or days. A ping response implies that at least the Internet connection and router/firewall at the remote location are up – so any claimed problems would be further downstream. The built in ping in Windows does not have a date or time stamp, so it isn’t useful for longer term tracking. And a lot of intermittent problems really do need to be monitored over periods of time.

e.g. fping google.com -D -T -l -c -L F:\temp\log.txt

will get you results something like this (on screen and in the text file):

2017/11/10 17:13:57.780 : Reply[1] from google.com: bytes=32 time=42.6 ms TTL=56
2017/11/10 17:13:58.824 : Reply[2] from google.com: bytes=32 time=43.1 ms TTL=56
2017/11/10 17:13:59.869 : Reply[3] from google.com: bytes=32 time=43.1 ms TTL=56

There are a ton of other options… but, I haven’t played with them much.

Originally from http://www.kwakkelflap.com (which is now just serving ads). By Wouter Dhondt – Thanks Wouter!

I haven’t been able to find any new address for the tool, so, I’ve decided to host a copy.

This is the latest version that I could find (using archive.org from September 2015): fping300.zip

Not to be confused with fping on Linux (Unix?) which is entirely different.

RIP yi.org?

September 29th, 2017

My very first host name was a free dynamic host name provided by the good folks behind yi.org. It was a classic story line, kid with his first always-on internet connection (cable, later ADSL) wants to host a web page on the world wide web! This ran well for me for well over a decade… until today.

I don’t even recall how I ended up with this free service, they were one of the first, but probably not the first dynamic host name provider. I suspect I just liked that “yi.org” was many characters shorter than the alternatives.

Unfortunately, as best as I can tell, the site has been shut down. My best guess, based on the snapshots available from archive.org, is that sometime in 2015 it all went away.

Apparently, they have graciously maintained DNS service for the host names that were in their system (I suspect updates have been disabled for a while). I wouldn’t have noticed that the site was gone if the static IP address for my little site finally had to be revised.

Long ago I deprecated that host name in favour of a lotus-land.ca address (of which I have control); however, I still type the shorter “yi.org” address often.

RIP yi.org, and thanks for all the great lookups!