9509 Commits

Author SHA1 Message Date
Patrick Dube
110f66ff13 CLOUDSTACK-8793 Added project id to create vpn customer gateway, and to the impl of list vpn connections and list vpn customer gateways 2015-10-28 09:58:37 -04:00
Aaron Brady
cacac14425 Pass LbProtocol down to the HAProxyConfigurator 2015-10-28 13:42:51 +00:00
Remi Bergsma
7d46b2ee56 Merge pull request #895 from SudharmaJain/cs-8911
CLOUDSTACK-8911: VM start job got stuck in loop looking for suitable host

VM instance creation job get stuck in the loop, when VMs require local storage there are host that reached max guest limit and remain hosts does have storage available.  This happens because the hosts that reach the max guest limit were not getting added to the avoid list and hence the cluster.

Verified the fix on my local setup.

Repro Steps:
1. Take an environment with single cluster and 2 hosts.
2. change the max guest limit for the hypervisor such that on one host max guest limit should reach.
3. change thresholds so that other host should not have enough storage. If required create a VM for sufficient bigger disk.
4. Now deploy a VM with local storage.
5. cluster will never be put in the avoid set and job will keep looking for suitable host.
6. once we increase the max guest limit, VM will deploy or will fail if there is a lack of storage.

* pr/895:
  CLOUDSTACK-8911: VM start job got stuck in loop looking for suitable host

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-28 11:29:32 +01:00
Kshitij Kansal
301ea330ce CLOUDSTACk-9002: VM deployment is successful even when dhcp entry command fails - Fixed 2015-10-28 11:51:25 +05:30
Remi Bergsma
535ab51b9a Merge pull request #849 from karuturi/CLOUDSTACK-8816-take2
Cloudstack-8816 some of the events do not have resource uuidsThe key objects in the context map are sometimes String and sometimes object. This causes missing uuids when an entity put in the context map with key entity.toString is queried with key entity

Testing:
manually tested by deploying a vm and checked that the created events in rabbitmq now has uuids.
events before and after the change are update at https://issues.apache.org/jira/browse/CLOUDSTACK-8816?focusedCommentId=14805239

unittests
```
$ mvn -pl :cloud-api test -Dtest=CallContextTest
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.cloudstack.context.CallContextTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.152 sec - in org.apache.cloudstack.context.CallContextTest

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.445 s
[INFO] Finished at: 2015-09-18T14:58:53+05:30
[INFO] Final Memory: 55M/448M
[INFO] ------------------------------------------------------------------------
```

* pr/849:
  CLOUDSTACK-8816 added missing events
  CLOUDSTACK-8816: fixed missing resource uuid in delete network cmd
  CLOUDSTACK-8816: fixed missing resource uuid in destroy vm event
  Cloudstack-8816: Fixed missing resource uuid in delete snapshot events
  CLOUDSTACK-8816: some of the events do not have resource uuids
  CLOUDSTACK-8816: some of the events do not have resource uuids

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-27 19:26:44 +01:00
Nick Livens
645f8758df CLOUDSTACK-8832 : Update Nuage VSP plugin to work with Nuage VSP release 3.2 2015-10-27 14:56:57 +01:00
Rajani Karuturi
fb4e6ed6ba Merge pull request #894 from sudhansu7/CLOUDSTACK-8917
CLOUDSTACK-8917 : Instance tab takes long time to load with 12K Vmsmodified sql that is used for retrieving vm count .

In load test environment listVirtualmachine takes 8-11 sec to load. This environment has around 12k active VMs. Total number of rows is 190K.

Performance bottleneck in listVirtualmachine command is fetching the count and distinct vms.
{noformat}
        // search vm details by ids
        Pair<List<UserVmJoinVO>, Integer> uniqueVmPair = _userVmJoinDao.searchAndCount(sc, searchFilter);
        Integer count = uniqueVmPair.second();
{noformat}

 This takes 95% of the total time.

To fetch the count and distinct vms we are using below sqls.

 Query 1:
{noformat}
SELECT DISTINCT(user_vm_view.id) FROM user_vm_view WHERE user_vm_view.account_type != 5  AND user_vm_view.display_vm = 1  AND user_vm_view.removed IS NULL  ORDER BY user_vm_view.id ASC  LIMIT 0, 20
 {noformat}

 Query 2:

select count(distinct id) from user_vm_view WHERE user_vm_view.account_type != 5  AND user_vm_view.display_vm = 1  AND user_vm_view.removed IS NULL

Query 2 is a problematic query.

If we rewrite the query as mentioned below then it will be ~2x faster.

select count(*) from (select distinct id from user_vm_view WHERE user_vm_view.account_type != 5  AND user_vm_view.display_vm = 1  AND user_vm_view.removed IS NULL) as temp;

Mysql Test result:

With 134 active Vms (total rows 349)
mysql> select count(*) from vm_instance;
+----------+
| count(*) |
+----------+
|      349 |
+----------+
1 row in set (0.00 sec)
mysql> select count(*) from user_vm_view;
+----------+
| count(*) |
+----------+
|      135 |
+----------+
1 row in set (0.02 sec)
mysql> select count(distinct id) from user_vm_view WHERE user_vm_view.account_type != 5  AND user_vm_view.display_vm = 1  AND user_vm_view.removed IS NULL;
+--------------------+
| count(distinct id) |
+--------------------+
|                134 |
+--------------------+
1 row in set (0.02 sec)

mysql> select count(*) from (select distinct id from user_vm_view WHERE user_vm_view.account_type != 5  AND user_vm_view.display_vm = 1  AND user_vm_view.removed IS NULL) as temp;
+----------+
| count(*) |
+----------+
|      134 |
+----------+
1 row in set (0.01 sec)

With 14326 active Vms (total rows 195660)

mysql> select count(*) from vm_instance;
+----------+
| count(*) |
+----------+
|   195660 |
+----------+
1 row in set (0.04 sec)
mysql> select count(*) from user_vm_view;
+----------+
| count(*) |
+----------+
|    41313 |
+----------+
1 row in set (4.55 sec)
mysql> select count(distinct id) from user_vm_view WHERE user_vm_view.account_type != 5  AND user_vm_view.display_vm = 1  AND user_vm_view.removed IS NULL;
+--------------------+
| count(distinct id) |
+--------------------+
|              14326 |
+--------------------+
1 row in set (7.39 sec)

mysql> select count(*) from (select distinct id from user_vm_view WHERE user_vm_view.account_type != 5  AND user_vm_view.display_vm = 1  AND user_vm_view.removed IS NULL) as temp;
+----------+
| count(*) |
+----------+
|    14326 |
+----------+
1 row in set (2.08 sec)

UI test Results:
Before:
![screen shot 2015-09-28 at 2 19 55 pm](https://cloud.githubusercontent.com/assets/1062642/10133848/66af7c40-65fe-11e5-9ef5-ec6489c0fc06.png)

After
![screen shot 2015-09-28 at 2 33 38 pm](https://cloud.githubusercontent.com/assets/1062642/10133852/6f512c9a-65fe-11e5-9ea1-890cf84d02b4.png)

* pr/894:
  CLOUDSTACK-8917 : Instance tab takes long time to load with 12K active VM (total vms: 190K)

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-10-27 12:17:55 +05:30
Rajani Karuturi
29dfeac857 CLOUDSTACK-8889: delete volume doesnt decrement primary store resource count
Primary Storage count for an account does not decrease when a Data Disk
is deleted belonging to the account unless the VM to which volume
belonged is destroyed

The resource counts are updated even before the disk is actually deleted
resulting in the same value.
Moved the resource counts updation to after the expunge operation as
thats when the disk is actually deleted.

all the tests in test/integration/component/test_ps_limits.py now pass
2015-10-26 17:40:27 +05:30
Remi Bergsma
f241455a63 Merge pull request #954 from ustcweizhou/CLOUDSTACK-8964
CLOUDSTACK-8964: Can't create template or volume from snapshot on KVM

* pr/954:
  CLOUDSTACK-8964: Can't create template or volume from snapshot

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-26 06:31:23 +01:00
Rajani Karuturi
2c0af1f867 CLOUDSTACK-8816 added missing events
added missing events for createUser, updateUser, createAccount apis.
2015-10-26 09:15:33 +05:30
Rajani Karuturi
3ff7bf771d CLOUDSTACK-8816: fixed missing resource uuid in delete network cmd
*events before*
| management-server.AsyncJobEvent.submit.None.*
| cloudstack-events | 7             |
{"cmdInfo":"{\"id\":\"edf0a16b-54cd-442e-b644-4af933f34229\",\"response\":\"json\",\"ctxDetails\":\"{\\\"interface
com.cloud.network.Network\\\":\\\"edf0a16b-54cd-442e-b644-4af933f34229\\\"}\",\"cmdEventType\":\"NETWORK.DELETE\",\"ctxUserId\":\"2\",\"httpmethod\":\"GET\",\"_\":\"1444805881664\",\"uuid\":\"edf0a16b-54cd-442e-b644-4af933f34229\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"1378\"}","instanceType":"None","jobId":"f7cbf481-49d0-423b-8661-5d3d678f4b96","status":"IN_PROGRESS","processStatus":"0","commandEventType":"NETWORK.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
| 736           | string           | True        | |
management-server.AsyncJobEvent.complete.None.* | cloudstack-events | 6
|
{"cmdInfo":"{\"id\":\"edf0a16b-54cd-442e-b644-4af933f34229\",\"response\":\"json\",\"ctxDetails\":\"{\\\"interface
com.cloud.network.Network\\\":\\\"edf0a16b-54cd-442e-b644-4af933f34229\\\"}\",\"cmdEventType\":\"NETWORK.DELETE\",\"ctxUserId\":\"2\",\"httpmethod\":\"GET\",\"_\":\"1444805881664\",\"uuid\":\"edf0a16b-54cd-442e-b644-4af933f34229\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"1378\"}","instanceType":"None","jobId":"f7cbf481-49d0-423b-8661-5d3d678f4b96","status":"FAILED","processStatus":"0","commandEventType":"NETWORK.DELETE","resultCode":"530","command":"org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd","jobResult":"org.apache.cloudstack.api.response.ExceptionResponse/null/{\"uuidList\":[],\"errorcode\":530,\"errortext\":\"Failed
to delete
network\"}","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
| 884           | string           | True        |

*events after*
|
management-server.AsyncJobEvent.submit.Network.5eccaece-a789-4b93-99c2-8b731ab6e328
| cloudstack-events | 1             |
{"cmdInfo":"{\"id\":\"5eccaece-a789-4b93-99c2-8b731ab6e328\",\"response\":\"json\",\"ctxDetails\":\"{\\\"interface
com.cloud.network.Network\\\":\\\"5eccaece-a789-4b93-99c2-8b731ab6e328\\\"}\",\"cmdEventType\":\"NETWORK.DELETE\",\"ctxUserId\":\"2\",\"httpmethod\":\"GET\",\"_\":\"1444814151636\",\"uuid\":\"5eccaece-a789-4b93-99c2-8b731ab6e328\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"1424\"}","instanceType":"Network","instanceUuid":"5eccaece-a789-4b93-99c2-8b731ab6e328","jobId":"d2cd4b27-acbd-4e56-867f-fe67ebde8261","status":"IN_PROGRESS","processStatus":"0","commandEventType":"NETWORK.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
| 793           | string           | False       |
|
management-server.AsyncJobEvent.complete.Network.5eccaece-a789-4b93-99c2-8b731ab6e328
| cloudstack-events | 0             |
{"cmdInfo":"{\"id\":\"5eccaece-a789-4b93-99c2-8b731ab6e328\",\"response\":\"json\",\"ctxDetails\":\"{\\\"interface
com.cloud.network.Network\\\":\\\"5eccaece-a789-4b93-99c2-8b731ab6e328\\\"}\",\"cmdEventType\":\"NETWORK.DELETE\",\"ctxUserId\":\"2\",\"httpmethod\":\"GET\",\"_\":\"1444814151636\",\"uuid\":\"5eccaece-a789-4b93-99c2-8b731ab6e328\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"1424\"}","instanceType":"Network","instanceUuid":"5eccaece-a789-4b93-99c2-8b731ab6e328","jobId":"d2cd4b27-acbd-4e56-867f-fe67ebde8261","status":"SUCCEEDED","processStatus":"0","commandEventType":"NETWORK.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd","jobResult":"org.apache.cloudstack.api.response.SuccessResponse/null/{\"success\":true}","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
| 880           | string           | False       |
2015-10-26 09:15:33 +05:30
Rajani Karuturi
ec03473c23 CLOUDSTACK-8816: fixed missing resource uuid in destroy vm event
*event before*
| management-server.AsyncJobEvent.complete.VirtualMachine.*
| cloudstack-events | 2             |
{"cmdInfo":"{\"response\":\"json\",\"id\":\"ba45d114-9844-4123-8dc6-7ae46d10581a\",\"ctxDetails\":\"{\\\"interface
com.cloud.vm.VirtualMachine\\\":\\\"ba45d114-9844-4123-8dc6-7ae46d10581a\\\"}\",\"cmdEventType\":\"VM.DESTROY\",\"ctxUserId\":\"2\",\"httpmethod\":\"GET\",\"_\":\"1444812001047\",\"uuid\":\"ba45d114-9844-4123-8dc6-7ae46d10581a\",\"ctxAccountId\":\"2\",\"expunge\":\"true\",\"ctxStartEventId\":\"1395\"}","instanceType":"VirtualMachine","jobId":"b46faa05-7b3a-4dbf-a78d-fbc7c66c3ce3","status":"SUCCEEDED","processStatus":"0","commandEventType":"VM.DESTROY","resultCode":"0","command":"org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin","jobResult":"org.apache.cloudstack.api.response.UserVmResponse/null/{\"securitygroup\":[],\"nic\":[],\"tags\":[],\"affinitygroup\":[]}","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
| 894           | string           | True        |

*event after*
|
management-server.AsyncJobEvent.complete.VirtualMachine.22e3bf71-91c8-4b18-a57e-af02d79dbb58
| cloudstack-events | 0             |
{"cmdInfo":"{\"response\":\"json\",\"id\":\"22e3bf71-91c8-4b18-a57e-af02d79dbb58\",\"ctxDetails\":\"{\\\"interface
com.cloud.vm.VirtualMachine\\\":\\\"22e3bf71-91c8-4b18-a57e-af02d79dbb58\\\"}\",\"cmdEventType\":\"VM.DESTROY\",\"ctxUserId\":\"2\",\"httpmethod\":\"GET\",\"_\":\"1444813240169\",\"uuid\":\"22e3bf71-91c8-4b18-a57e-af02d79dbb58\",\"ctxAccountId\":\"2\",\"expunge\":\"true\",\"ctxStartEventId\":\"1418\"}","instanceType":"VirtualMachine","instanceUuid":"22e3bf71-91c8-4b18-a57e-af02d79dbb58","jobId":"256ca2e7-de05-4b33-b32a-aa8567f05160","status":"SUCCEEDED","processStatus":"0","commandEventType":"VM.DESTROY","resultCode":"0","command":"org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin","jobResult":"org.apache.cloudstack.api.response.UserVmResponse/null/{\"securitygroup\":[],\"nic\":[],\"tags\":[],\"affinitygroup\":[]}","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
| 948           | string           | False       |
2015-10-26 09:15:32 +05:30
Rajani Karuturi
04554ddd24 Cloudstack-8816: Fixed missing resource uuid in delete snapshot events
*event before*

| management-server.AsyncJobEvent.complete.Snapshot.*
| cloudstack-events | 26            |
{"cmdInfo":"{\"id\":\"2ebabd8f-0b34-4461-8071-0917c231ca49\",\"response\":\"json\",\"ctxDetails\":\"{\\\"interface
com.cloud.storage.Snapshot\\\":\\\"2ebabd8f-0b34-4461-8071-0917c231ca49\\\"}\",\"cmdEventType\":\"SNAPSHOT.DELETE\",\"ctxUserId\":\"2\",\"httpmethod\":\"GET\",\"_\":\"1444803845320\",\"uuid\":\"2ebabd8f-0b34-4461-8071-0917c231ca49\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"1345\"}","instanceType":"Snapshot","jobId":"fab1feaf-3b4f-4158-b332-a78e43fee5e0","status":"SUCCEEDED","processStatus":"0","commandEventType":"SNAPSHOT.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotCmd","jobResult":"org.apache.cloudstack.api.response.SuccessResponse/null/{\"success\":true}","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}

*After*

|
management-server.AsyncJobEvent.complete.Snapshot.f25ad748-2fe3-4911-b40c-4698425c8a2f
| cloudstack-events | 0             |
{"cmdInfo":"{\"id\":\"f25ad748-2fe3-4911-b40c-4698425c8a2f\",\"response\":\"json\",\"ctxDetails\":\"{\\\"interface
com.cloud.storage.Snapshot\\\":\\\"f25ad748-2fe3-4911-b40c-4698425c8a2f\\\"}\",\"cmdEventType\":\"SNAPSHOT.DELETE\",\"ctxUserId\":\"2\",\"httpmethod\":\"GET\",\"_\":\"1444806612980\",\"uuid\":\"f25ad748-2fe3-4911-b40c-4698425c8a2f\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"1388\"}","instanceType":"Snapshot","instanceUuid":"f25ad748-2fe3-4911-b40c-4698425c8a2f","jobId":"69849909-9082-481c-b8ee-9ddc1608fe8d","status":"SUCCEEDED","processStatus":"0","commandEventType":"SNAPSHOT.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.user.snapshot.DeleteSnapshotCmd","jobResult":"org.apache.cloudstack.api.response.SuccessResponse/null/{\"success\":true}","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
| 886           | string           | True        |
2015-10-26 09:15:32 +05:30
Rajani Karuturi
242b87dbfb CLOUDSTACK-8816: some of the events do not have resource uuids
uuid is missing in the first event of VM create as the entity is just
created and never put in the Context.
Added the entity uuid to context on successful creation.
2015-10-26 09:15:32 +05:30
Wei Zhou
ccf5d75cfb CLOUDSTACK-8964: Can't create template or volume from snapshot
This issue happens on KVM.
Normally the SSVM will process the CopyCommand from snapshot to template.
However, Ovm3HypervisorGuru chooses a KVM hypervisor to process the CopyCommand.
This is obviously wrong.
2015-10-23 22:02:24 +02:00
Funs Kessen
1022883749 FIX: Ovm3 physical network traffic labels to work.
The labeling was broken. Only labels assigned at zone creation
were used, changing labels was not working. Tested with changing
a label and checking it.

As a bonus fixed the consistency of KVM in Dutch compared to other
traffic labels in dutch and copied in the OVM3 translated label
in other languages.
2015-10-22 11:57:42 +02:00
Nitin Kumar Maharana
adcd23d15a CLOUDSTACK-8962: Dedicated cluster is used for virtual routers that belong to non-dedicated account
Earlier the deployment planner was not handling the case of virtual routers.(In Explicit Dedication)
It was only handling for all instance VMs/user VMs.
Added code for checking the case of Virtual Routers.
2015-10-22 12:57:56 +05:30
Rajani Karuturi
79b7391c22 CLOUDSTACK-8923: Create storage network IP range failed
transaction.getConnection() was wrapped in try-with-resource.
This caused transaction to get closed even before its committed.
2015-10-16 09:42:48 +05:30
Wilder Rodrigues
cd8c3e97f5 CLOUDSTACK-8934 - Fix the AclIP class to make it configure the default FW policy 2015-10-11 14:57:33 +02:00
Wilder Rodrigues
b4dc392bfd CLOUDSTACK-8934 - Fix default EGRESS rules for isolated networks
- The default is Accept and will be changed based on the configuration of the offering.

CLOUDSTACK-8934 - The default egress is set as Deny in the router.

   - We had to change it on the Java side in order to make the apply it once the default is defined as allowed on the net offering
2015-10-11 14:57:32 +02:00
SudharmaJain
2d34036a28 CLOUDSTACK-8911: VM start job got stuck in loop looking for suitable host 2015-09-30 09:05:25 +05:30
Sudhansu
c28a58a8ff CLOUDSTACK-8917 : Instance tab takes long time to load with 12K active VM (total vms: 190K)
modified sql that is used for retrieving vm count .
2015-09-28 16:24:26 +05:30
Remi Bergsma
7d5555429b Merge pull request #851 from SudharmaJain/cs-8864
CLOUDSTACK-8864: Not able to add TCP port forwarding rule in VPN for specific ports

Setting port forwarding rules for port 500,1701 and 4500 after enabling VPN, gives the error message "The range specified, xxxx, conflicts with rule xxxx which has xxxx." This happens because the rules added for vpn doesn't have a matching condition to allow port forwarding rules.

Added a unit test to verify the detectRulesConflict function of FirewallManagerImpl.

* pr/851:
  CLOUDSTACK-8864: Not able to add TCP port forwarding rule in VPN for specific ports

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-09-24 15:35:16 +02:00
Boris Schrijver
80cb3adad0 Added isEmpty() check to prevent nullPointerException. 2015-09-23 15:07:16 +02:00
Bharat Kumar
3c61746dce CLOUDSTACK-89027 Restart Network fails in EIP/ELB zone 2015-09-23 16:19:37 +05:30
SudharmaJain
96c38bf491 CLOUDSTACK-8864: Not able to add TCP port forwarding rule in VPN for specific ports 2015-09-19 23:40:21 +05:30
Boris Schrijver
5a3a3d641d Added unit tests for checkIfPodIsDeletable() and checkIfZoneIsDeletable(). 2015-09-17 17:28:29 +02:00
Boris Schrijver
b31e64d49d Refactored checkIfZoneIsDeletable().
It now makes use of various DAO DB function instead of running a self build query.
2015-09-16 22:19:14 +02:00
Boris Schrijver
45861fa1b7 Refactored checkIfPodIsDeletable().
It now makes use of various DAO DB function instead of running a self build query.
2015-09-16 22:14:05 +02:00
Maneesha.P
07ee5e9bf0 CLOUDSTACK-8866 : restart.retry.interval is being used instead of migrate.retry.interval during host maintenance 2015-09-16 12:52:34 +05:30
Wido den Hollander
06cefaf493 Merge pull request #804 from anshul1886/CLOUDSTACK-8834
CLOUDSTACK-8834: Fixed unable to download Template , when in multi zonesWe were listing image stores by zone id which was resulting in listing of only one image store
If in that image store its download state is not DOWNLOADED then download template is failing

* pr/804:
  CLOUDSTACK-8834: Fixed unable to download Template , when in multi zones We were listing image stores by zone id which was resulting in listing of only one image store If in that image store its download state is not DOWNLOADED then download template is failing

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-09-15 11:10:41 +02:00
Rajani Karuturi
ac80a2df50 Merge pull request #808 from karuturi/CLOUDSTACK-8835
CLOUDSTACK-8835: Added alerts incase of template download failureAuthored-By: @sanjaytripathi
Reviewed-By: @devdeep

* pr/808:
  CLOUDSTACK-8835: Added alerts incase of template download failure

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-09-14 09:47:46 +05:30
Rajani Karuturi
f888e93e44 Merge pull request #782 from karuturi/CLOUDSTACK-8816
Cloudstack 8816 entityuuid missing in some of the eventsIn some of the events generated, entity uuid was missing making it difficult to find the entity. Fixed the same.

Tested it on rabbitmq instance.
There are the events before after the fix:

Before
--------------------------------------------------------------------------------

routing_key: management-server.ActionEvent.ACCOUNT-DELETE.Account.*
exchange: cloudstack-events
message_count: 2
payload:
{"eventDateTime":"2015-09-04 17:59:24 +0530","status":"Scheduled","description":"deleting User test4 (id: 28) and accountId \u003d 28","event":"ACCOUNT.DELETE","Account":"c09e2e81-8edc-4c27-b072-25005b522b63","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}

payload_bytes: 304
payload_encoding: string
redelivered: False

--------------------------------------------------------------------------------

routing_key: management-server.AsyncJobEvent.complete.Account.*
exchange: cloudstack-events
message_count: 0
payload: {"cmdInfo":"{\"id\":\"9dd3abc2-3f8b-4852-aa60-a74b234acb13\",\"response\":\"json\",\"sessionkey\":\"5ig1ItP2_5v-mgY4cVJbJN5hw_w\",\"ctxDetails\":\"
{\\\"interface com.cloud.user.Account\\\":\\\"9dd3abc2-3f8b-4852-aa60-a74b234acb13\\\"}

\",\"cmdEventType\":\"ACCOUNT.DELETE\",\"expires\":\"2015-09-07T11:11:56+0000\",\"ctxUserId\":\"2\",\"signatureversion\":\"3\",\"httpmethod\":\"GET\",\"uuid\":\"9dd3abc2-3f8b-4852-aa60-a74b234acb13\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"447\"}","instanceType":"Account","jobId":"5004989d-0cde-4922-8afa-66bf38b75ea7","status":"SUCCEEDED","processStatus":"0","commandEventType":"ACCOUNT.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd","jobResult":"org.apache.cloudstack.api.response.SuccessResponse/null/
{\"success\":true}

","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
payload_bytes: 914
payload_encoding: string
redelivered: False

--------------------------------------------------------------------------------

After
--------------------------------------------------------------------------------

 routing_key: management-server.ActionEvent.ACCOUNT-DELETE.Account.e5e2db91-414d-484c-99d5-c4e265c14ad8
exchange: cloudstack-events
message_count: 13
payload: {"eventDateTime":"2015-09-07 17:32:26 +0530","status":"Completed","description":"Successfully completed deleting account. Account Id: 45","event":"ACCOUNT.DELETE","entityuuid":"e5e2db91-414d-484c-99d5-c4e265c14ad8","entity":"com.cloud.user.Account","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
payload_bytes: 344
payload_encoding: string
redelivered: True

--------------------------------------------------------------------------------

routing_key: management-server.AsyncJobEvent.complete.Account.e5e2db91-414d-484c-99d5-c4e265c14ad8
exchange: cloudstack-events
message_count: 12
payload: {"cmdInfo":"{\"id\":\"e5e2db91-414d-484c-99d5-c4e265c14ad8\",\"response\":\"json\",\"sessionkey\":\"8AJVbn8HIpg5LZ_VaVfSPs_QN2k\",\"ctxDetails\":\"{\\\"interface com.cloud.user.Account\\\":\\\"e5e2db91-414d-484c-99d5-c4e265c14ad8\\\"}\",\"cmdEventType\":\"ACCOUNT.DELETE\",\"expires\":\"2015-09-07T12:17:42+0000\",\"ctxUserId\":\"2\",\"signatureversion\":\"3\",\"httpmethod\":\"GET\",\"uuid\":\"e5e2db91-414d-484c-99d5-c4e265c14ad8\",\"ctxAccountId\":\"2\",\"ctxStartEventId\":\"465\"}","instanceType":"Account","instanceUuid":"e5e2db91-414d-484c-99d5-c4e265c14ad8","jobId":"0bb08486-6d9f-4e9f-bfef-b7463c42e71b","status":"SUCCEEDED","processStatus":"0","commandEventType":"ACCOUNT.DELETE","resultCode":"0","command":"org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd","jobResult":"org.apache.cloudstack.api.response.SuccessResponse/null/{\"success\":true}","account":"bd73dc2e-35c0-11e5-b094-d4ae52cb9af0","user":"bd7ea748-35c0-11e5-b094-d4ae52cb9af0"}
payload_bytes: 968
payload_encoding: string
redelivered: True

--------------------------------------------------------------------------------

* pr/782:
  CLOUDSTACK-8816 Systemvm reboot event doesnt have uuids. Fixed the same
  CLOUDSTACK-8816: Project UUID is not showing for some of operations in RabbitMQ.
  CLOUDSTACK-8816: entity uuid missing in create network event
  CLOUDSTACK-8816: instance uuid is missing in events for delete account
  CLOUDSTACK-8816 Fixed entityUuid missing in some cases is events

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-09-14 09:42:44 +05:30
Ashwin Swaroop
221624d2fe CLOUDSTACK-8817: listFirewallRules response JSON startport/endport not an int 2015-09-13 14:45:44 -04:00
Wido den Hollander
5812f714fc Merge pull request #807 from karuturi/createuser-unittests
unittests to verify empty password is not allowed during account/user create

* pr/807:
  unittests to verify empty password is not allowed during account create

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-09-11 14:54:47 +02:00
Sanjay Tripathi
9a35f87d37 CLOUDSTACK-8835: Added alerts incase of template download failure
Reviewed-By: Devdeep
2015-09-11 16:07:35 +05:30
Rajani Karuturi
1865433e69 unittests to verify empty password is not allowed during account create 2015-09-11 15:52:38 +05:30
Anshul Gangwar
8511d62b5a CLOUDSTACK-8834: Fixed unable to download Template , when in multi zones
We were listing image stores by zone id which was resulting in listing of only one image store
If in that image store its download state is not DOWNLOADED then download template is failing
2015-09-11 15:30:39 +05:30
Wei Zhou
e586d11a1b CLOUDSTACK-5863: Add unit tests for take volume snapshot 2015-09-10 08:01:33 +02:00
Wei Zhou
007f5b842d CLOUDSTACK-5863: Add unit tests for create/delete/revert volume snapshot 2015-09-10 08:01:30 +02:00
wilderrodrigues
2d90f18b82 Merge pull request #788 from ekholabs/fix/iso_net-CLOUDSTACK-8814
CLOUDSTACK-8814 - Refactoring the configuration of Routers and VPC routers nicsHi there,

I refactored the configureDefaultNics() method in order to split the implementations for Routers and VPC Routers.

The following tests were executed:

* test_vm_life_cycle
* test_routers
* test_vpc_router_nics
* test_vpc_routers
* test_vpc_offerings

@remibergsma @bhaisaab @koushik-das @miguelaferreira @DaanHoogland @karuturi , could you please have a look/test this PR?

Thanks in advance.

Cheers,
Wilder

* pr/788:
  CLOUDSTACK-8814 - Refactoring the configuration of Routers and VPC routers nics

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-09-09 14:08:15 +02:00
Rajani Karuturi
358846492a CLOUDSTACK-8816 Systemvm reboot event doesnt have uuids. Fixed the same
Also removed duplicate console proxy event thats generated both for ssvm
and console proxy reboot.
2015-09-09 14:23:29 +05:30
Rajani Karuturi
3bd53521e8 CLOUDSTACK-8816: Project UUID is not showing for some of operations in RabbitMQ. 2015-09-09 14:23:28 +05:30
Rajani Karuturi
37c3451aae CLOUDSTACK-8816 Fixed entityUuid missing in some cases is events
context parameters is Map<object,Object>. This has been used
interchangeably with String and class object resulting the param value
not being available in some cases if its put with object and get with
class.forName()

made the put and get consistent by using Object as key everywhere.
2015-09-09 14:23:27 +05:30
Rajani Karuturi
76c81a7bd9 Merge pull request #685 from kansal/CLOUDSTACK-8727
CLOUDSTACK-8727: API call listVirtualMachines returns same keypairCurrently the user can register same key with different names. Upon listing the VM's the name which got registered first is being returned and not the actual one. Anyhow this behavior is rare and not good. I have added a UNIQUE constraint on the ssh_keypairs table and also made sure that the previous registered keys(with duplicates) get deleted.

* pr/685:
  CLOUDSTACK-8727: API call listVirtualMachines returns same keypair: Corrected and test cases added

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-09-09 10:39:43 +05:30
Kshitij Kansal
82fbd80193 CLOUDSTACK-8727: API call listVirtualMachines returns same keypair: Corrected and test cases added 2015-09-09 09:35:48 +05:30
wilderrodrigues
5e9e9b84fb CLOUDSTACK-8814 - Refactoring the configuration of Routers and VPC routers nics 2015-09-08 15:34:29 +02:00
Rohit Yadav
30792532ed Merge pull request #777 from resmo/fix/config-string
config: doc: add default value 10 in description for vm.network.nic.max.secondary.ipaddresses

value hardcoded by commit 4925b9f6a126454215531998c461bf376ac6ab67

* pr/777:
  config: doc: add default value 10 in description for vm.network.nic.max.secondary.ipaddresses

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-08 16:55:51 +05:30
Remi Bergsma
b6e212f2c7 Merge pull request #773 from ekholabs/fix/vpc_nic-CLOUDSTACK-8759
CLOUDSTACK-8759 - Destroying VPC router results in a new unusable VPC routerHow to reproduce the problem:

1. Stop/Destroy the VPC router
2. Add a virtual machine to one of the VPC tier - it will trigger a VPC router creation
3. Router is created, but the NICs are not configured

How to recover without this fix:
1. Stop/destroy the VPC router and restart the VPC

Side effects: private gateways could be misconfigured.

Root cause:

In the VpcNetworkHelperImpl.configureDefaultNics() method, the guest network nic was added in the map prior to the control and public NICs. The order in the map should not matter, however in the LibvirtComputingResource.createVifs() method, there is a logic that relies on the device index - the array index - in order to create the  control nic. I advise a refactor on the data model in order to be able to identify the NIC type instead of relying in the array index.

An integration test was added to cover the fix:
* test_vpc_router_nics.py

Environment:
Management Server running on CentOS 7.1
KVM host running on CentOS 7.1
CloudStack Agent/Common 4.6.0-SNAPSHOT

Executing the test:

```
nosetests --with-marvin --marvin-config=/data/shared/marvin/mct-zone2-kvm2-ISOLATED.cfg -s -a tags=advanced,required_hardware=true component/test_vpc_router_nics.py
```

Remark: during the SSH there might be stack traces on the console due to the connection retry. It takes some time to get the PF rules in place and reach the VMs. So, just let the test run until the end.

```
Test results:

Create a vpc with two networks with two vms in each network ... === TestName: test_01_VPC_nics_after_destroy | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 1 test in 774.020s

OK
/tmp//MarvinLogs/test_vpc_router_nics_VH6E9S/results.txt (END)
```

* pr/773:
  CLOUDSTACK-8759 - Fix guets nic allocation
  CLOUDSTACK-8759 - Adding a marvin test in order to cover the fix
  CLOUDSTACK-8759 - The guest nic has to be added after the control nic

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-09-05 11:41:18 +02:00
Rajani Karuturi
5881035e7b Merge pull request #755 from karuturi/CLOUDSTACK-8647-2
Cloudstack:8647 LDAP Trust AD and AutoimportToday, CloudStack can automatically import LDAP users based on the configuration to a domain or an account. However, any new users in LDAP aren't automatically reflected. The admin has to manually import them again.
This feature enables admin to map LDAP group/OU to a CloudStack domain and any changes are reflected in ACS as well.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/WIP%3A+LDAP%3A+Trust+AD+and+Auto+Import

testcases output:
```
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running groovy.org.apache.cloudstack.ldap.NoLdapUserMatchingQueryExceptionSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.216 sec - in groovy.org.apache.cloudstack.ldap.NoLdapUserMatchingQueryExceptionSpec
Running groovy.org.apache.cloudstack.ldap.LdapManagerImplSpec
log4j:WARN No appenders could be found for logger (org.apache.cloudstack.ldap.LdapManagerImpl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
using type:
using type: null
using type: TEST
using type: TEST TEST
using name:
using name: null
using accountType: -1
using accountType: 1
using accountType: 3
using accountType: 4
using accountType: 5
using accountType: 6
using accountType: 20000
using accountType: -500000
Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.387 sec - in groovy.org.apache.cloudstack.ldap.LdapManagerImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapListUsersCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.041 sec - in groovy.org.apache.cloudstack.ldap.LdapListUsersCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapAddConfigurationCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec - in groovy.org.apache.cloudstack.ldap.LdapAddConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserSpec
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.021 sec - in groovy.org.apache.cloudstack.ldap.LdapUserSpec
Running groovy.org.apache.cloudstack.ldap.LdapAuthenticatorSpec
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 sec - in groovy.org.apache.cloudstack.ldap.LdapAuthenticatorSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationVOSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationVOSpec
Running groovy.org.apache.cloudstack.ldap.OpenLdapUserManagerSpec
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.094 sec - in groovy.org.apache.cloudstack.ldap.OpenLdapUserManagerSpec
Running groovy.org.apache.cloudstack.ldap.LdapDeleteConfigurationCmdSpec
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec - in groovy.org.apache.cloudstack.ldap.LdapDeleteConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserResponseSpec
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapUserResponseSpec
Running groovy.org.apache.cloudstack.ldap.LdapUserManagerFactorySpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.027 sec - in groovy.org.apache.cloudstack.ldap.LdapUserManagerFactorySpec
Running groovy.org.apache.cloudstack.ldap.ADLdapUserManagerImplSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 sec - in groovy.org.apache.cloudstack.ldap.ADLdapUserManagerImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapCreateAccountCmdSpec
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.168 sec - in groovy.org.apache.cloudstack.ldap.LdapCreateAccountCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapImportUsersCmdSpec
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec - in groovy.org.apache.cloudstack.ldap.LdapImportUsersCmdSpec
Running groovy.org.apache.cloudstack.ldap.LinkDomainToLdapCmdSpec
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec - in groovy.org.apache.cloudstack.ldap.LinkDomainToLdapCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapSearchUserCmdSpec
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 sec - in groovy.org.apache.cloudstack.ldap.LdapSearchUserCmdSpec
Running groovy.org.apache.cloudstack.ldap.LdapListConfigurationCmdSpec
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapListConfigurationCmdSpec
Running groovy.org.apache.cloudstack.ldap.NoSuchLdapUserExceptionSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec - in groovy.org.apache.cloudstack.ldap.NoSuchLdapUserExceptionSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationResponseSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationResponseSpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationSpec
asserting for provider configuration: openldap
asserting for provider configuration: microsoftad
asserting for provider configuration:
asserting for provider configuration:
asserting for provider configuration: xyz
asserting for provider configuration: MicrosoftAd
asserting for provider configuration: OpenLdap
asserting for provider configuration: MicrosoftAD
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.053 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationSpec
Running groovy.org.apache.cloudstack.ldap.LdapContextFactorySpec
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.099 sec - in groovy.org.apache.cloudstack.ldap.LdapContextFactorySpec
Running groovy.org.apache.cloudstack.ldap.LdapConfigurationDaoImplSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.027 sec - in groovy.org.apache.cloudstack.ldap.LdapConfigurationDaoImplSpec
Running groovy.org.apache.cloudstack.ldap.LdapUtilsSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in groovy.org.apache.cloudstack.ldap.LdapUtilsSpec

Results :

Tests run: 156, Failures: 0, Errors: 0, Skipped: 0
```

* pr/755:
  CLOUDSTACK-8647: linkdomaintoldap shouldnt fail when createuseraccount fails
  CLOUDSTACK-8647 removed duplicate key in create sql of ldap_trust_map
  CLOUDSTACK-8647: string formatting
  CLOUDSTACK-8647: updated with review comments
  CLOUDSTACK-8647: unittests for LdapAuthenticatorSpec
  CLOUDSTACK-8647: formatted LdapAuthenticatorSpec
  CLOUDSTACK-8647: UI for trust AD feature
  CLOUDSTACK-8647 added unittests for new methods  in ldapmanager
  CLOUDSTACK-8647 unittests for LinkDomainToLdap api command
  CLOUDSTACK-8647: fixed unittests
  CLOUDSTACK-8647 support for assigning and admin to linked ldap domain
  CLOUDSTACK-8647 added nested group enabled config in ldap
  CLOUDSTACK-8647 added account_type to the linkDomainToLdap API
  CLOUDSTACK-8647 changed the authentication flow
  CLOUDSTACK-8647 added new api linkLdapToDomain
  CLOUDSTACK-8647: added cmd and response class for the new api

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-09-05 09:30:17 +05:30