CLOUDSTACK-9088: Update the description for migrateVirtualMachineWithVolume apihttps://issues.apache.org/jira/browse/CLOUDSTACK-9088
* pr/1126:
CLOUDSTACK-9088: Update the description for migrateVirtualMachineWithVolume api.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
[CLOUDSTACK-9218]Test to verify restart network after master VR destroyedPlease refer CLOUDSTACK-9218 for more details
Test Results:
===========
Test restarting RvR network without cleanup after destroying master VR ... === TestName: test_restart_ntwk_MVR_destroyed | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 1 test in 581.194s
OK
* pr/1323:
Added new test to verify restart network after destorying master VR Bug-Id: CLOUDSTACK-9218
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Fixing an issue in Marvin around creating a template from a snapshotThis fixes the following ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-9354
The problem was that Marvin was requiring you to pass in the "ispublic" parameter when creating a template from a snapshot.
As the ticket notes, this issue was introduced by the following commit: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=bbe0fc4be9527d51820b067a602886003991db4d
The solution I've provided is simply to check if the "ispublic" property is in the dictionary before referencing it.
* pr/1501:
CLOUDSTACK-9354 - Fixing an issue in Marvin around creating a template from a snapshot (if “is public” is not provided, there was a problem)
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag
RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.
* pr/1200:
CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
* 4.8:
CLOUDSTACK-9172 Added cross zones check to delete template and iso
Check the existence of 'forceencap' parameter before use
systemvm: set default umask 022 in injectkeys.sh
* 4.7:
CLOUDSTACK-9172 Added cross zones check to delete template and iso
Check the existence of 'forceencap' parameter before use
systemvm: set default umask 022 in injectkeys.sh
CLOUDSTACK-9172 Added cross zones check to delete template and isoAdded a check to ignore the zoneid, in the delete template UI, if the template is cross zones.
reference : CLOUDSTACK-9172
* pr/1505:
CLOUDSTACK-9172 Added cross zones check to delete template and iso
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Check the existence of 'forceencap' parameter before useCheck the existence of 'forceencap' parameter before use.
Error seen:
```
Traceback (most recent call last):
File "/opt/cloud/bin/update_config.py", line 140, in <module>
process_file()
File "/opt/cloud/bin/update_config.py", line 54, in process_file
finish_config()
File "/opt/cloud/bin/update_config.py", line 44, in finish_config
returncode = configure.main(sys.argv)
File "/opt/cloud/bin/configure.py", line 1003, in main
vpns.process()
File "/opt/cloud/bin/configure.py", line 488, in process
self.configure_ipsec(self.dbag[vpn])
File "/opt/cloud/bin/configure.py", line 544, in configure_ipsec
file.addeq(" forceencaps=%s" % CsHelper.bool_to_yn(obj['encap']))
KeyError: 'encap'
```
* pr/1402:
Check the existence of 'forceencap' parameter before use
Signed-off-by: Will Stevens <williamstevens@gmail.com>
systemvm: preserve file permissions, set default umask- In injectkeys.sh which is used to inject new public keys everytime cloudstack
starts; while copying files preserve the mode/ownership. This ensures the
scripts have same mode bits as originally configured in the iso file
- The default umask of 0022 is set in Ubuntu and other packages. Set the same
in case of CentOS startup scripts
cc @abhinandanprateek @wido @remibergsma @DaanHoogland @jburwell
* pr/1420:
systemvm: set default umask 022 in injectkeys.sh
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9100: ISO.CREATE/TEMPLATE.CREATE event missing for usage_event by template sync thread
If there is a Management server restart while template is in downloading or installing state. Template Sync does not push event into usage_event table.
I have verified the fix manually. Here is a snapshot.
I have registered 4 templates. template id 207 and 208(ISO.CREATE event is missing) before applying the fix. and template id 209 and 210 after applying the fix.
Repro Steps (3 cases)
==========
Case - 1 (private template)
-------------
1. register a private template/iso.
2. restart management server when template is in downloading state.
3. After management server restart, template_store_ref entry is removed if download was not yet completed.
4. on next management server restart, if download would have completed, template_store_ref entry will get populated, but TEMPLATE.CREATE event is missing in usage_event.
Case - 2 (Public template)
--------------------------------
1. register public template.
2. restart management server when in downloading state.
3. after restart template download reinitiates.
4. template goes to ready state, but there is no usage event.
case -3 (public/private template)
---------------------------------
1. register a template
2. restart management server when template is in installing state.
3. after restart template goes to ready, but there is no usage event.
* pr/1157:
CLOUDSTACK-9100: ISO.CREATE/TEMPLATE.CREATE event missing for usage_event by template sync thread
Signed-off-by: Koushik Das <koushik@apache.org>
* 4.8:
CLOUDSTACK-9272: No option in UI to add GSLB with service type "HTTP"
CLOUDSTACK-9270: UI alignment gone bad in multiple places - VM Instance, Network, Egress rules
* 4.7:
CLOUDSTACK-9272: No option in UI to add GSLB with service type "HTTP"
CLOUDSTACK-9270: UI alignment gone bad in multiple places - VM Instance, Network, Egress rules
CLOUDSTACK-9272: No option in UI to add GSLB with service type "HTTP"Steps to Repro:
============
Go to Regions -> Local -> View GSLB -> Add GSLB
Click on the service type dropdown
Observe http is missing. Please see the attached snapshot.
Expected Behaviour:
================
As it supports http also, So http should be in the list.
Actual Behaviour:
==============
http is missing from the list.
Fix:
===
It supports http also. Added http to the list.
Snapshot:
========
<img width="531" alt="gslb-http-missing-nitin" src="https://cloud.githubusercontent.com/assets/12583725/12772818/21513dc0-ca5b-11e5-822e-e2dd2426da65.png">
* pr/1399:
CLOUDSTACK-9272: No option in UI to add GSLB with service type "HTTP"
Signed-off-by: Koushik Das <koushik@apache.org>
CLOUDSTACK-9268: Display VM in Load balancing rule in UISteps of Repro:
=============
1:Create VMs
2:Make LoadBalancing rule in GUI
Name:WWW
PrivatePort:80
PublicPort:80
Add VMs:some VMs
Expected Result:
==============
The VMs which has been already assigned is should not be listed when you add the VM to an existing rule.
Actual Result:
===========
The VMs which has been already assigned is still being listed when you add the VM to an existing rule.
Fix:
===
Added jsonObj to newly created row in multiedit.js to stop listing the same VM again.
* pr/1394:
CLOUDSTACK-9268: Display VM in Load balancing rule in UI
Signed-off-by: Koushik Das <koushik@apache.org>
Removed unnecessary code from getGuestOsType in CitrixResourceBaseConsidering that all mapping of Guest OS Names to their respective hypervisor compatible types is made thorugh accessing a database, we've decided to remove a bit of code in the XcpOssResource class which was doing that same thing for 2 different OS's (both of which ARE in the database). That has led us to a bunch of unused parameters in the getGuestOsType method from its superclass, which we've also decided to remove. Test cases were added for four different possibilities for the platformEmulator String: one for a null String, one for a blank String, one for an empy String and one for a random case with a valid String.
* pr/1262:
Remove test cases duplicated code.
Removed unnecessary code from getGuestOsType in CitrixResourceBase
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9251: Fix issue in scale VM to dynamic service offeringThis reverts commit 9c4162ac7f451fc3e2155418dcfff224c8c08a4a and 16baa1289b7de383e98d0070717b3f1873fa2db3
Before change: exception when change compute offering (to dynamic service offering) on UI
After change: succeed
* pr/1363:
Fix issue in scale VM to dynamic service offering
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Removed unused code from com.cloud.api.ApiServer**Removed \_ from variables names**: private variables with \_ at the beginning is common in C++ but not in Java.
**Removed unused code from ApiServer:**
- com.cloud.api.ApiServer.getPluggableServices(): unused method;
- com.cloud.api.ApiServer.getApiAccessCheckers(): unused method;
**Methods and variables access level reviewed:**
- com.cloud.api.ApiServer.handleAsyncJobPublishEvent(String, String ,Object): this method was private but the annotation @MessageHandler requests public methods, as can be seen in org.apache.cloudstack.framework.messagebus.MessageDispatcher.buildHandlerMethodCache(Class\<?\>), which searches methods with the @MessageHandler annotation and changes
it to be accessible (setAccessible(true)). Thus, there is no reason for handleAsyncJobPublishEvent be a private method and lead some other dev to wrong conclusions about the use of the method;
- Global variables and methods called just by this class (ApiServer) were changed to private.
**Changed variables and methods from static to non-static (if possible):** as some variables/methods are used just by one object of this class, instantiated by Spring, they were changed to non-static.
With that, calls from com.cloud.api.ApiServlet.ApiServlet() that used static methods from ApiServer, were changed from ApiServer.\<staticMethodName\> to \_apiServer.\<methodName\> that refers to the org.apache.cloudstack.api.ApiServerService interface. Thus, methods com.cloud.api.ApiServer.getJSONContentType() and com.cloud.api.ApiServer.isSecureSessionCookieEnabled() had to be added in the interface (org.apache.cloudstack.api.ApiServerService, interface implemented by class ApiServer).
* pr/1263:
The goal of this PR is to review com.cloud.api.ApiServer class, with the following actions:
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Fixed Profiler's unit tests bugs.### **Problem:**
The TestProfiler class was using Java Thread methods to test the
Profiler's functionality. That was causing the tests to fail sometimes
since the JVM's thread priority could be low on some OS.
### **Fix:**
Using PowerMockito to mock the System calls, the threads could be
removed. This makes the tests considerably faster, OS independent and
still guarantees the correct implementation of the Profiler class.
The changes on the Profiler's class was only to shorten the class's line
size by not assigning the return value to a variable returning it
straight out.
* pr/1445:
Fixed changes to match code conventions
Fixed Profiler's unit tests bugs.
Signed-off-by: Will Stevens <williamstevens@gmail.com>
following actions:
Removed “_” in beginning of global variables names:
Variables was changed from “_<variablename>” to “<variablename>”, as
this convension (private veriables with “_”) is common in C++ but not in
Java.
Removed unused code from ApiServer:
- com.cloud.api.ApiServer.getPluggableServices():
Unused method.
- com.cloud.api.ApiServer.getApiAccessCheckers():
Unused method.
Methods and variables access level reviewed:
- com.cloud.api.ApiServer.handleAsyncJobPublishEvent(String, String,
Object):
This method was private but the annotation @MessageHandler requests
public methods, as can be seen in
org.apache.cloudstack.framework.messagebus.MessageDispatcher.buildHandlerMethodCache(Class<?>),
which searches methods with the @MessageHandler annotation and changes
it to accessible (“setAccessible(true)”). Thus, there is no reason for
handleAsyncJobPublishEvent be a private method.
- Global variables and methods called just by this class (ApiServer)
were changed to private.
Changed variables and methods from static to non static (if possible):
As some variables/methods are used just by one object of this class
(instantiated by springer), they were changed to non static.
With that, calls from com.cloud.api.ApiServlet.ApiServlet() that used
static methods from ApiServer, was changed from
ApiServer.<staticMethodName> to _apiServer.<methodName> that refers to
the org.apache.cloudstack.api.ApiServerService interface. Thus, methods
com.cloud.api.ApiServer.getJSONContentType() and
com.cloud.api.ApiServer.isSecureSessionCookieEnabled() had to be
included in the interface (org.apache.cloudstack.api.ApiServerService,
interface implemented by class ApiServer).
However, com.cloud.api.ApiServer.isEncodeApiResponse() was keept static,
as its call hierarchy would have to be changed (more than planed for
this PR).
speedup iptables by prefetching the variables-- This PR is replacing speedup iptables setup #1449
-- Squashing commits and cleanup
PR against 4.7 as discussed with Remi Bergsma. This will speed up the iptables creation on the virtual router.
Testing showed the following:
with current code:
root@kvm704:~# time /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh vr_cfg.sh 169.254.1.176 -c /var/cache/cloud/VR-12f28879-de7e-44d2-8dbe-b93a04bd3ba4.cfg
real 2m56.401s
user 0m0.012s
sys 0m0.012s
modified version:
root@kvm704:~# time /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh vr_cfg.sh 169.254.1.176 -c /var/cache/cloud/VR-12f28879-de7e-44d2-8dbe-b93a04bd3ba4.cfg
real 1m35.762s
user 0m0.020s
sys 0m0.004s
* pr/1487:
speedup iptables by prefetching the variables
Signed-off-by: Will Stevens <williamstevens@gmail.com>
Cloudstack-9285 exception log additionAfter discussion with @miguelaferreira on the previous PR related to Cloudstack-9285, we decided on adding additional exception logging for this issue.
After adding it, the logs look like this in our lab:
2016-04-07 15:44:03,298 WARN [cloud.agent.Agent] (Agent-Handler-1:null) (logid:7225632a) NIO Connection Exception com.cloud.utils.exception.NioConnectionException: Connection closed with -1 on reading size. <<-- new exception logging
2016-04-07 15:44:03,298 INFO [cloud.agent.Agent] (Agent-Handler-1:null) (logid:7225632a) Attempted to connect to the server, but received an unexpected exception, trying again... << --original logging from previous PR.
* pr/1479:
Additional exception logging for Cloudstack-9285
Signed-off-by: Will Stevens <williamstevens@gmail.com>
SecurityGroupRulesCmd code cleanupWrote a test and cleaned some duplicate code with the objective to evaluate the jenkins pull request process at builds.a.o
worthwhile to keep, IMHO.
* pr/1287:
SecurityGroupRulesCmd code cleanup review comments handled
deal with PMD warnings
code cleanup
security rules test
remove autogenerated pydev files
Signed-off-by: Koushik Das <koushik@apache.org>
CLOUDSTACK-9297 - Reworked logic in StorageSystemSnapshotStrategy and XenserverSnapshotStrategyThe ticket this PR fixes was opened because KVM-specific code had been added to the StorageSystemSnapshotStrategy class and that class' canHandle method was only prepared to handle managed storage being used with XenServer (and a case was hit for KVM that triggered a CloudRuntimeException to be thrown).
To solve the problem, I moved the KVM logic to the default snapshot strategy class, which is (unfortunately) named XenserverSnapshotStrategy.
I plan to rename XenserverSnapshotStrategy to something like DefaultSnapshotStrategy in 4.9.
My guess is that when XenserverSnapshotStrategy was originally written, it was written only for XenServer, but has since that time had its usage increased to support other hypervisors (with non-managed storage).
* pr/1441:
CLOUDSTACK-9297: delete snapshot without id is failing with Unable to determine the storage pool of the snapshot
Signed-off-by: Will Stevens <williamstevens@gmail.com>
[CLOUDSTACK-9215]Test to verify vm deployment in vpc tier if nic type is Vmxnet3Please check bug CLOUDSTACK-9215 for more details.
Test Results:
==========
Test to create vpc tier with nic type as Vmxnet3 ... === TestName: test_01_create_tier_Vmxnet3 | Status : SUCCESS ===
ok
----------------------------------------------------------------------
Ran 1 test in 591.630s
OK
* pr/1316:
New marvin test to validate CLOUDSTACK-9215 Bug-Id: CLOUDSTACK-9215
Signed-off-by: Will Stevens <williamstevens@gmail.com>
travis: increase build verbosityOutputs for modules that fail or succeed with unit tests results.
Based on the issue raised in https://github.com/apache/cloudstack/pull/1466 this PR aims at increasing Travis build output so we can know which unit test fail.
/cc @swill @nvazquez -- let me know if we just allow outputting everything will help? I had restricted the output as including all of them would disallow viewing the tabular final integration/marvin tests at the end (only possible by viewing raw output of each travis job).
* pr/1481:
travis: increase build verbosity
Signed-off-by: Will Stevens <williamstevens@gmail.com>
This fixes a typographical error in UI that did not previously send fetchLatest
flag in the listCapacity API request.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9333: Exclude clusters from OVF operationsJIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9333
## Introduction
In some environments there is a need to exclude certain VMware clusters from performing OVF operations. This operations are part of:
* create template
* create volume snaphsot
* copy template, volume, images from primary storage to secondary storage
* migrate volume
* participate when a template gets cached over to primary storage.
In ESX/ESXi, OVF operations are low priority and bound to a single CPU and most likely get throttled to certain IOPS and network limits.
If the hypervisor chosen for OVF operations is weak or overloaded this results in significantly longer execution of such OVF command and therefore degraded performance of underlying CloudStack API call.
### Proposed solution
It is proposed to add a way to exclude hosts from selected clusters for OVF operations.
To exclude a cluster, would be necessary to insert a record in <code>cluster_details</code> specifying property **vmware.exclude_from_ovf** in this way: (supposing we want to exclude cluster X)
| cluster_id | name| value |
|:-------------:|:-------------:|:-------------:|
|X|vmware.exclude_from_ovf|true|
* pr/1457:
CLOUDSTACK-9333: Exclude clusters for OVF operations
Signed-off-by: Will Stevens <williamstevens@gmail.com>
CLOUDSTACK-9174: A deleted account results in NPEWhen an account is deleted from cloudstack for which quota is still
being calculated and if the quota reaches minimum threshold then
quota service will try to alert the user. This results in NPE and is
fixed by excluding such accounts from alerting and other quota related
mechanisms.
* pr/1254:
CLOUDSTACK-9174: A deleted account results in NPE
Signed-off-by: Will Stevens <williamstevens@gmail.com>
travis: Fix simulator tests and optimize default global configs- Migrate to trusty based Travis VMs
- Increase tests across five build matrices
- Fix xunit-reader output, include time
- Fix pip/python usage, pkg installation
- Build CloudStack in parallel with -T4
- Deploy database with optimized global settings
cc @runseb @swill @wido @DaanHoogland
* pr/1461:
travis: Fix simulator tests and optimize default global configs
Signed-off-by: Will Stevens <williamstevens@gmail.com>
