Document - contributions welcome - for CLONE or UPGRADE to clusters

[RADSr]

I'd like to get feedback/additions/improvements on the below:

I'm working on capturing some notes as I'm in the midst of an upgrade adventure and think I may make a document for myself/Community/support to use when doing this kind of work.

Text from my initial plan below, but note that I'm combining two ideas because I've used them in conjunction - 1) upgrade and 2) cluster cloning

My initial ask is for things to include ( pitfalls, gotchas, tick lists, etc. ) when doing something like this.

1.  * not sure where this belongs
   1. Inspector Tool
      1. Support had me run this before upgrading PROD, but not the DEV clone.   
      2. Documentation says Sev 1 issues can interfere w/ the upgrade
      3. So probably resolve all Sev 1 issues before upgrading the clone
      4. Why the ease of ignoring this when updating from the cloud admin console?
          
2. Cluster cloning
   1. Turn off schedules!
      1. This is tremendously important because...
         1. Clones maintain not only the schedule but also the email settings so you'll get two of everything so 
   2. Change email settings
      1. It *looks* like the *default* tenant configurations update w/ the target cluster, but the existing tenant does not 
         1. I'm past the point where I can confirm this w/ my current project
   3. Re-add users in the cloud admin console
      1. Apparently if you have access to a cluster in cloud admin you do *not* automatically have access to it's clone - even if the clone target is an existing cluster to which you did have access.
   4. Data agent
      1. For a clone only I think you just need to regenerate the .auth file
      2. For an upgrade 
         1. Install new data agent 
         2. Generate new .auth file  ( do we need to create a new data agent data source in the IA UI? ) 
         3. Swear a lot when it doesn't work
         4. TBD
         5. If creating a new data source data agent update the data source data agent drop down 
   5. Kind of a cool note - it looks like the clone keeps the "last run" metadata so incremental loads will run from the last run of the clone source.
      1. Needs confirmation
         
         

=================== UPGRADE PLAN SAMPLE ==========================

 Upgrade Plan

1. Back up current DEV
   1. Tenant
   2. Parquet files for Incremental_Only_Snapshot_Tables schema
   3. Local data files
      
      
2. Delete all objects from DEV
   
   
3. B/U and Restore PROD to DEV
   1. Parquet files for Incremental_Only_Snapshot_Tables schema
   2. Local data files
      1.      At this point can we continue to run incremental loads or do we need an initial full load?
         1. How will this affect snapshots?
      2. 
         
         
   3. Upgrade DEV to 2023.4.0
      1. Test Incremental loads
      2. Test full loads
      3. A/B testing DEV to PROD
         
         
   4. Upgrade PROD
      
      
   5. Restore DEV  from B/U in step 1
      
      

Celebratory drinks and dinner!

[RADSr]

Adding for the "clone" checklist:    Make sure the newly-cloned environment is pointing at the correct storage.

I just encountered an issue where I uploaded a file to a cloned environment but found the file unavailable for use in creating a table.   

The issue was that somewhere deep in the configuration ( i.e. I had ask support to check and fix ) the clone retained the pointer to the original physical file storage, but the create table dialog points - properly - to the new cloned cluster physical storage. 

TL/DR - after creating a clone upload a test file and check it's availability for table creation.