Signed-off-by: Mice Xia <mice_xia@tcloudcomputing.com>
This commit is contained in:
Sebastien Goasguen 2012-09-10 10:33:23 +08:00 committed by Alex Huang
parent 7467db1c00
commit abfebbfa4e
5 changed files with 0 additions and 2232 deletions

View File

@ -1,80 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
CloudStack Migration: 1.0.x to 2.1.x
How it works:
There are four major steps to migrating a 1.0.x system's data to version 2.1.x of the CloudStack:
1. Users in the 1.0.x system need to be re-created in the 2.1.x system
2. Public IPs that are allocated in the 1.0.x system need to be allocated in the 2.1.x sytem
3. Port forwarding and load balancer rules that were created in the 1.0.x system need to be re-created in the 2.1.x system
4. Virtual machines and the data on their root/data disks need to be migrated
To accomplish steps 1, 2, and 3, the CloudStack Migration tool automatically reads information from 1.0.x system's database and re-creates the data in the 2.1.x system through a combination of API calls and direct SQL inserts.
To accomplish step 4, the tool creates a direct link between 1.0.x storage servers and 2.1.x XenServers, and copies volume data using the XenServer API.
The overall process should take between 15-30 minutes per VM, depending on the speed of your private network and the size of the volumes involved.
What you need:
1. A running 1.0.x system that has one zone and one pod.
2. Necessary hardware for the 2.1.x system: one or more management servers, and one or more XenServers that are all on the same public network as the 1.0.x system.
* The 2.1.x management server must be able to access the 1.0.x management server's database, as well as the 1.0.x system's storage servers.
3. The 10to21Upgrade.tgz package.
How to run the migration tool:
1. If you DO NOT have a 2.1.x system installed and running: Do a fresh 2.1.x install (please refer to the admin guide for instructions), taking into account the following special instructions:
* Before you add any XenServer host, add one public IP range into the system with exactly two public IPs; these must be unallocated in the 1.0.x system.
* After adding all of your XenServer hosts in the UI, verify that the secondary storage VM and console proxy VM started. Then, add remainining public IPs as a second IP range.
2. Register a bootable ISO and note down its database ID (you will need this for step 6). The OS of the ISO doesn't matter.
If you already have a bootable ISO in the 2.1 system, you can use its database ID in step 6.
* If you have no preference about which ISO to use, simply enter the following URL to register an Ubuntu 10.04 ISO:
http://localhost:8096/client/api?command=registerIso&bootable=true&zoneid=1&ispublic=true&name=Ubuntu&displayText=Ubuntu&url=http://ftp.ucsb.edu/pub/mirrors/linux/ubuntu/10.04/ubuntu-10.04.1-desktop-amd64.iso&ostypeid=59
* Else, use the following API command (replacing variables as necessary):
http://localhost:8096/client/api/?command=registerIso&bootable=true&zoneid=1&ispublic=true&name=ISO_NAME&displayText=ISO_DISPLAY_TEXT&url=ISO_URL&ostypeid=ISO_OS_TYPE_ID
* To determine the ISO_OS_TYPE_ID, run the following API command and find the ID that corresponds to the OS of the ISO:
http://localhost:8096/client/api/?command=listOsTypes
3. For every service offering in the 1.0.x system:
* Make sure there is a service offering in the 2.1.x system with the same cpu #, cpu speed, and RAM size.
* Make sure there is a disk offering in the 2.1.x system with the same disk size. 1.0.x allowed for creating service offerings with disk sizes that had an arbitrary number of MB.
However, in 2.1.x, disk offerings must be created in multiples of 1 GB. If there is a service offering in the 1.0.x system with a disk size that is not a
multiple of 1 Gb (1024 MB), create a disk offering in the 2.1.x system that is the 1.0.x disk size rounded to the next GB. For example, a disk size of 2000 MB in 1.0.x
will correspond to a disk offering with size 2 GB in the 2.1.x system.
4. Install Python on the 2.1.x management server, if it isn't already installed. Version 2.4 or above is required.
5. Download 10to21Upgrade.tgz to any folder on the 2.1.x management server, and uncompress it.
6. Fill out upgrade.properties. Instructions about various fields are included in the file itself.
7. If you DO have a 2.1.x system installed and running:
* Add a new public IP range in the 2.1.x system that corresponds to the public IP range in the 1.0.x system.
The public IP ranges that already exist in the 2.1.x system must not overlap with the IP range in the 1.0.x system.
* Run "python upgrade.py publicips". This will immediately allocate the public IPs of all users in the 1.0.x system, so that existing 2.1.x users can't allocate them.
8. Run "python upgrade.py" on the 2.1.x management server. Status information will be printed out to the console.
* If there is an error, please contact CloudStack Support and send us the migration log. By default, this file is called "migrationLog" and is in the same directory as upgrade.py.
* After the cause for an error has been resolved, you can run upgrade.py again from the beginning; it will skip over any work that has already been done.
* If you would like to re-enable a user on 1.0.x system, simply stop all of the user's VMs that have been migrated on the 2.1.x system, and start the user's VMs on the 1.0.x system.

View File

@ -1,200 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
import gettext
import xmlrpclib
import httplib
import socket
translation = gettext.translation('xen-xm', fallback = True)
API_VERSION_1_1 = '1.1'
API_VERSION_1_2 = '1.2'
class Failure(Exception):
def __init__(self, details):
self.details = details
def __str__(self):
try:
return str(self.details)
except Exception, exn:
import sys
print >>sys.stderr, exn
return "Xen-API failure: %s" % str(self.details)
def _details_map(self):
return dict([(str(i), self.details[i])
for i in range(len(self.details))])
_RECONNECT_AND_RETRY = (lambda _ : ())
class UDSHTTPConnection(httplib.HTTPConnection):
"""HTTPConnection subclass to allow HTTP over Unix domain sockets. """
def connect(self):
path = self.host.replace("_", "/")
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
self.sock.connect(path)
class UDSHTTP(httplib.HTTP):
_connection_class = UDSHTTPConnection
class UDSTransport(xmlrpclib.Transport):
def __init__(self, use_datetime=0):
self._use_datetime = use_datetime
self._extra_headers=[]
def add_extra_header(self, key, value):
self._extra_headers += [ (key,value) ]
def make_connection(self, host):
return UDSHTTP(host)
def send_request(self, connection, handler, request_body):
connection.putrequest("POST", handler)
for key, value in self._extra_headers:
connection.putheader(key, value)
class Session(xmlrpclib.ServerProxy):
"""A server proxy and session manager for communicating with xapi using
the Xen-API.
Example:
session = Session('http://localhost/')
session.login_with_password('me', 'mypassword')
session.xenapi.VM.start(vm_uuid)
session.xenapi.session.logout()
"""
def __init__(self, uri, transport=None, encoding=None, verbose=0,
allow_none=1):
xmlrpclib.ServerProxy.__init__(self, uri, transport, encoding,
verbose, allow_none)
self.transport = transport
self._session = None
self.last_login_method = None
self.last_login_params = None
self.API_version = API_VERSION_1_1
def xenapi_request(self, methodname, params):
if methodname.startswith('login'):
self._login(methodname, params)
return None
elif methodname == 'logout' or methodname == 'session.logout':
self._logout()
return None
else:
retry_count = 0
while retry_count < 3:
full_params = (self._session,) + params
result = _parse_result(getattr(self, methodname)(*full_params))
if result == _RECONNECT_AND_RETRY:
retry_count += 1
if self.last_login_method:
self._login(self.last_login_method,
self.last_login_params)
else:
raise xmlrpclib.Fault(401, 'You must log in')
else:
return result
raise xmlrpclib.Fault(
500, 'Tried 3 times to get a valid session, but failed')
def _login(self, method, params):
result = _parse_result(getattr(self, 'session.%s' % method)(*params))
if result == _RECONNECT_AND_RETRY:
raise xmlrpclib.Fault(
500, 'Received SESSION_INVALID when logging in')
self._session = result
self.last_login_method = method
self.last_login_params = params
self.API_version = self._get_api_version()
def _logout(self):
try:
if self.last_login_method.startswith("slave_local"):
return _parse_result(self.session.local_logout(self._session))
else:
return _parse_result(self.session.logout(self._session))
finally:
self._session = None
self.last_login_method = None
self.last_login_params = None
self.API_version = API_VERSION_1_1
def _get_api_version(self):
pool = self.xenapi.pool.get_all()[0]
host = self.xenapi.pool.get_master(pool)
major = self.xenapi.host.get_API_version_major(host)
minor = self.xenapi.host.get_API_version_minor(host)
return "%s.%s"%(major,minor)
def __getattr__(self, name):
if name == 'handle':
return self._session
elif name == 'xenapi':
return _Dispatcher(self.API_version, self.xenapi_request, None)
elif name.startswith('login') or name.startswith('slave_local'):
return lambda *params: self._login(name, params)
else:
return xmlrpclib.ServerProxy.__getattr__(self, name)
def xapi_local():
return Session("http://_var_xapi_xapi/", transport=UDSTransport())
def _parse_result(result):
if type(result) != dict or 'Status' not in result:
raise xmlrpclib.Fault(500, 'Missing Status in response from server' + result)
if result['Status'] == 'Success':
if 'Value' in result:
return result['Value']
else:
raise xmlrpclib.Fault(500,
'Missing Value in response from server')
else:
if 'ErrorDescription' in result:
if result['ErrorDescription'][0] == 'SESSION_INVALID':
return _RECONNECT_AND_RETRY
else:
raise Failure(result['ErrorDescription'])
else:
raise xmlrpclib.Fault(
500, 'Missing ErrorDescription in response from server')
# Based upon _Method from xmlrpclib.
class _Dispatcher:
def __init__(self, API_version, send, name):
self.__API_version = API_version
self.__send = send
self.__name = name
def __repr__(self):
if self.__name:
return '<XenAPI._Dispatcher for %s>' % self.__name
else:
return '<XenAPI._Dispatcher>'
def __getattr__(self, name):
if self.__name is None:
return _Dispatcher(self.__API_version, self.__send, name)
else:
return _Dispatcher(self.__API_version, self.__send, "%s.%s" % (self.__name, name))
def __call__(self, *args):
return self.__send(self.__name, args)

View File

@ -1,238 +0,0 @@
# !/usr/bin/env bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# lu_share.sh -- makes all logical units (LUs) available over iSCSI, for a specified initiator IQN
# OpenSolaris
usage() {
printf "Usage: %s -i <initiator-iqn> [ -s | -u ]\n" $(basename $0) >&2
}
valid_target_name() { # <target-name>
echo $1 | grep ':lu:' >/dev/null
return $?
}
target_iqn_from_target_name() { # <target-name>
echo $1 | cut -d':' -f1,2,3
}
hg_from_initiator_iqn() { # <initiator-iqn>
echo $1
return 0
}
lu_name_from_target_name() { # <target-name>
echo $1 | cut -d':' -f5
}
view_entry_from_hg_and_lu_name() { # <host-group-name> <lu-name>
local hg=$1
local lu_name=$2
local view=
local last_view=
local last_hg=
for w in $(stmfadm list-view -l $lu_name)
do
case $w in
[0-9]*) last_view=$w
;;
esac
if [ "$w" == "$hg" ]
then
echo $last_view
return 0
fi
done
return 1
}
create_host_group() { # <initiator-iqn>
local hg=$1
local i_iqn=$2
local host_group=
local result=
result=$(stmfadm create-hg $hg 2>&1)
if [ $? -ne 0 ]
then
echo $result | grep "already exists" > /dev/null
if [ $? -ne 0 ]
then
printf "%s: create-hg %s failed due to %s\n" $(basename $0) $i_iqn $result >&2
return 11
fi
fi
result=$(stmfadm add-hg-member -g $hg $i_iqn 2>&1)
if [ $? -ne 0 ]
then
echo $result | grep "already exists" > /dev/null
if [ $? -ne 0 ]
then
printf "%s: unable to add %s due to %s\n" $(basename $0) $i_iqn $result >&2
return 12
fi
fi
return 0
}
add_view() { # <hg> <lu_name>
local i=1
local hg=$1
local lu=$2
while [ $i -lt 500 ]
do
local lun=$[ ( $RANDOM % 512 ) ]
local result=
result=$(stmfadm add-view -h $hg -n $lun $lu 2>&1)
if [ $? -eq 0 ]
then
printf "lun %s for luname %s\n" $lun $lu >&2
#stmfadm list-view -l $lu
#sbdadm list-lu
return 0
fi
echo $result | grep "view entry exists" > /dev/null
if [ $? -eq 0 ]
then
return 0
fi
echo $result | grep "LUN already in use" > /dev/null
if [ $? -ne 0 ]
then
echo $result
return 1
fi
let i=i+1
done
printf "Unable to add view after lots of tries\n" >&2
return 1
}
add_view_and_hg() { # <initiator_iqn> <lu_name>
local i_iqn=$1
local lu_name=$2
local hg="Migration"
local result=
if ! create_host_group $hg $i_iqn
then
printf "%s: create_host_group failed: %s %s\n" $(basename $0) $i_iqn $lu_name >&2
return 22
fi
if ! add_view $hg $lu_name
then
return 1
fi
return 0
}
remove_view() { # <initiator-iqn> <lu-name>
local lu_name=$1
local hg="Migration"
local view=$(view_entry_from_hg_and_lu_name $hg $lu_name)
if [ -n "$view" ]
then
local result=
result=$(stmfadm remove-view -l $lu_name $view 2>&1)
if [ $? -ne 0 ]
then
echo $result | grep "not found"
if [ $? -eq 0 ]
then
return 0
fi
echo $result | grep "no views found"
if [ $? -eq 0 ]
then
return 0
fi
printf "Unable to remove view due to: $result\n" >&2
return 5
fi
fi
return 0
}
# set -x
iflag=
sflag=
uflag=
while getopts 'sui:' OPTION
do
case $OPTION in
i) iflag=1
init_iqn="$OPTARG"
;;
s) sflag=1
;;
u) uflag=1
;;
*) usage
exit 2
;;
esac
done
if [ "$sflag$iflag" != "11" -a "$uflag" != "1" ]
then
usage
exit 3
fi
lu_names="$(stmfadm list-lu | cut -d":" -f2)"
for lu_name in $lu_names
do
if [ "$uflag" == "1" ]
then
remove_view $lu_name
if [ $? -gt 0 ]
then
printf "%s: remove_view failed: %s\n" $(basename $0) $lu_name >&2
exit 1
fi
else
if [ "$sflag" == "1" ]
then
add_view_and_hg $init_iqn $lu_name
if [ $? -gt 0 ]
then
printf "%s: add_view failed: %s\n" $(basename $0) $lu_name >&2
exit 1
fi
fi
fi
done
if [ "$uflag" == "1" ]
then
stmfadm delete-hg "Migration"
fi
exit 0

View File

@ -1,102 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
### Users to upgrade
# Specify the list of user IDs to upgrade as a comma separated list; i.e. 3, 4, 5
# This is optional; you can also run upgrade.py with a list of user IDs; i.e. "python upgrade.py 3 4 5".
USERS =
### Information about the 1.0.x system
# The management server IP
SRC_MANAGEMENT_SERVER_IP =
# The database username and password
SRC_DB_LOGIN =
SRC_DB_PASSWORD =
# A map between storage host IPs and root passwords
# Ex: 1.2.3.4:password1, 2.3.4.5:password2
STORAGE_HOST_PASSWORDS =
# The id of the zone
SRC_ZONE_ID = 1
### Information about the 2.1.x system
# The management server IP
DEST_MANAGEMENT_SERVER_IP = localhost
# The database username and password
DEST_DB_LOGIN =
DEST_DB_PASSWORD =
# The private IP and root password of one of the XenServers in the 2.1.x system
# Fill this section out only if all of your XenServers have the same root password
DEST_XENSERVER_IP =
DEST_XENSERVER_PASSWORD =
# A map between XenServer IPs in the 2.1.x system to passwords for each host
# I.e. 1.2.3.4:password1, 2.3.4.5:password2, 3.4.5.6:password3
# Fill this section out only if your XenServers have different root passwords
DEST_XENSERVER_PASSWORDS =
# A map between template IDs in the 1.0.x system to guest OS IDs in the 2.1.x system
# Should be in the format: [1.0 template ID]:[2.1 guest OS ID]. Ex: 3:12, 4:14, 5:64
# To find the ID that corresponds to a guest OS, refer to the output of the API command: http://localhost:8096/client/api/?command=listOsTypes
GUEST_OS_MAP =
# The id of the ISO you registered
DEST_ISO_ID = 201
# The id of the zone
DEST_ZONE_ID = 1
# The id of the default CentOS template
DEST_TEMPLATE_ID = 2
# The id of the default service offering
DEST_SERVICE_OFFERING_ID = 3
# The id of the default disk offering
DEST_DISK_OFFERING_ID = 5
# The id of the guest OS category that corresponds to Windows
DEST_WINDOWS_GUEST_OS_CATEGORY_ID = 6
### Misc. variables
# The location of the log file
LOG_FILE = ./migrationLog
# The location of the migrated users file
MIGRATED_ACCOUNTS_FILE = ./migratedAccounts
# The number of retries for async API commands
ASYNC_RETRIES = 20
# The time to pause between retries for async API commands
ASYNC_SLEEP_TIME = 30

File diff suppressed because it is too large Load Diff