29777 Commits

Author SHA1 Message Date
Ronald van Zantvoort
58bdb44135 SysVM: Cleanup and removal of old (and dangerous) config files
* ports.conf
* default & default-ssl sites
* SSL config in httpd.conf
* deprecated & dead setup_redundant_router in cloud-early-config
2016-06-07 13:03:10 +02:00
Ronald van Zantvoort
72e83085e1 SysVM cloud-early-config: Intermediate fix for SecStore & CORS
* Take setup from vhost.template rather than default(-ssl)
  * should move into Python CS code as well
* Move CORS setup to separate conf
* Modify vhost template to Optionally include the cors file
* Add NameVirtualHost to vhost template for feature parity with ports.conf
* Take setup from vhost.template rather than default(-ssl)
2016-06-07 13:03:10 +02:00
Ronald van Zantvoort
f49a4d944a VR cloud-early-config: Commonize Apache2 common setup 2016-06-07 13:03:10 +02:00
Ronald van Zantvoort
7e290979a9 VR cloud-early-config: Fix Apache2 alias cleanup 2016-06-07 13:03:10 +02:00
Ronald van Zantvoort
e32cd1303a VR: consistent SSL setup, vhost is not an example, but a template 2016-06-07 13:03:10 +02:00
Ronald van Zantvoort
f379df4bc2 VR CsConfig: reintroduce old get_dns() behaviour for redundant non-VPC's 2016-06-07 13:03:10 +02:00
Ronald van Zantvoort
d14a484374 VR CsAddress fixes:
* cleanup imports,
* fix to_str(),
* improve & fix service post_config logic
* don't arpPing when there's no gateway
2016-06-07 13:03:09 +02:00
Ronald van Zantvoort
6055ed6ed6 VR CsApp: Expose config to classes, move vhost confs to proper location, allow for multiple IP's per intf, sanitize servername, don't open port 53 if no DNS is foreseen 2016-06-07 13:03:09 +02:00
Ronald van Zantvoort
748bf43530 VR CsConfig: Add is_router(), is_dns(), has_dns(), has_metadata(), use_extdns(), fix get_dns() with use_extdns() 2016-06-07 13:03:09 +02:00
Ronald van Zantvoort
875379042e VR CsDhcp: allow multiple ranges & finite lease time (fixes CLOUDSTACK-8303) 2016-06-07 13:03:09 +02:00
Ronald van Zantvoort
2790d7a69b VR CsGuestNetwork obey useextdns 2016-06-07 13:03:09 +02:00
Ronald van Zantvoort
ed820bebe5 VR merge.py ipalias fix & dhcpconfig stub notification 2016-06-07 13:03:09 +02:00
Will Stevens
971c8a74e4 Merge pull request #1574 from nlivens/mysql_driver_issue
Make sure that the DB drivers are loaded before creating connectionsI've digged deeper, and found out that Tomcat is really specific in how it loads the JDBC drivers apparently.
If we would be using the standard JDBC connection pooling of Tomcat (tomcat-jdbc) instead of commons-dbcp, we would have the option to specify a "driverClassName" when creating our connection.
This is not the case for commons-dbcp, which we are using within ACS.

If you check an official example of Tomcat :
https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#Plain_Ol'_Java
or
https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#As_a_Resource

As you can see in the above examples, both of them specify the driverClassName.
In the underlying implementation of Tomcat, Tomcat will do ```Class.forName(driverClassName)``` which will trigger the auto-registration of the Driver.

Tomcat code :
```java
            if (driver==null) {
                if (log.isDebugEnabled()) {
                    log.debug("Instantiating driver using class: "+poolProperties.getDriverClassName()+" [url="+poolProperties.getUrl()+"]");
                }
                driver = (java.sql.Driver) Class.forName(poolProperties.getDriverClassName(),
                                                         true, PooledConnection.class.getClassLoader()
                                                         ).newInstance();
            }
```

* pr/1574:
  Make sure that the DB drivers are loaded before initiating connections

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-06-01 11:39:16 -04:00
Nick Livens
c9567495ab Make sure that the DB drivers are loaded before initiating connections 2016-05-31 14:39:43 +02:00
Will Stevens
bb9148b3cc Merge pull request #1570 from shapeblue/travis-ipmitool-remove
travis: use ipmitool from ubuntu repository@swill @pdion891 please review, merge this once Travis is green. The packages.shapeblue.com server is unreachable which is causing a wget command to fail. This fixes this by installing `ipmitool` from ubuntu repositories.

* pr/1570:
  travis: add one more smoke test
  travis: use ipmitool from ubuntu repository

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-30 08:47:34 -04:00
Will Stevens
6968b18a47 Merge pull request #1569 from pdion891/fix-vmware60
fix noredist build because of missing maven dependency of vmware 6.0 libfix noredist build for new vmware lib

* pr/1569:
  fix noredist build because of missing maven dependency of vmware 6.0 lib

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-30 00:07:20 -04:00
Rohit Yadav
620a9bb1a9 travis: add one more smoke test
Adds a login smoke test to Travis
2016-05-29 16:25:53 +05:30
Rohit Yadav
91c9fc9f6e travis: use ipmitool from ubuntu repository
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-29 16:14:28 +05:30
Pierre-Luc Dion
7914cdc2be fix noredist build because of missing maven dependency of vmware 6.0 lib 2016-05-28 11:03:11 -04:00
Will Stevens
123e9f81ab fixed a variable name broken by merge conflicts 2016-05-27 16:32:20 -04:00
Will Stevens
237bd46d55 Merge pull request #1423 from syed/ipsec-start-bug
[CLOUDSTACK-9296] Start ipsec for client VPNThis fix starts the IPSEC daemon when enabling client side vpn

* pr/1423:
  [CLOUDSTACK-9296] Start ipsec for client VPN

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:21:16 -04:00
Will Stevens
75c8a55014 Merge pull request #1251 from koushik-das/CLOUDSTACK-9180
CLOUDSTACK-9180: Optimize concurrent VM deployment operation on same network

Check if VR needs to be allocated for a given network and only acquire lock if required

Refer to the bug for details.

* pr/1251:
  CLOUDSTACK-9180: Optimize concurrent VM deployment operation on same network Check if VR needs to be allocated for a given network and only acquire lock if required

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:20:49 -04:00
Will Stevens
121b3d6403 Merge pull request #1567 from exoscale/CLOUDSTACK-9238
CLOUDSTACK-9238: Fix URL length to 2048 for all url fields in VOI will update the PR to add max field length in the API commands too

* pr/1567:
  API: update url field max length
  not needed on host table
  Fix URL length to 2048 for all url fields in VO

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:20:22 -04:00
Will Stevens
20f779298d merging 4.8 -> master 2016-05-27 15:19:33 -04:00
Will Stevens
744f9d5615 Merge release branch 4.7 to 4.8
* 4.7:
  CLOUDSTACK-9376: Restrict listTemplates API with filter=all for root admin
  CLOUDSTACK-9369: Restrict default login to ldap/native users
  Add lsb-release dependency to mgmt server and agent on Debian/Ubuntu.
  Emit template UUID and class type over event bus when deleting templates.
2016-05-27 15:03:42 -04:00
Will Stevens
9275ba27f3 Merge pull request #1565 from greenqloud/pr-lsb-release-4.7
Add lsb-release dependency to mgmt server and agent on Debian/Ubuntu.New version of #1412, based on the 4.7 branch.

* pr/1565:
  Add lsb-release dependency to mgmt server and agent on Debian/Ubuntu.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:02:46 -04:00
Will Stevens
cd5f3320c8 Merge pull request #1564 from greenqloud/pr-emit-template-uuid-on-delete-4.7
Emit template UUID and class type over event bus when deleting templatesNew version of #1378 for the 4.7b branch instead of 4.6.

* pr/1564:
  Emit template UUID and class type over event bus when deleting templates.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:02:17 -04:00
Rohit Yadav
0cb60a72fe CLOUDSTACK-9376: Restrict listTemplates API with filter=all for root admin
Restricts use of listemplates API with templatefilter=all for root admin only.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-27 15:01:15 -04:00
Rohit Yadav
566e7d9fac CLOUDSTACK-9369: Restrict default login to ldap/native users
- Restricts default login auth handler to ldap and native-cloudstack users
- Refactors and create re-usable method to find domain by id/path
- Adds unit test for refactored method in DomainManagerImpl
- Adds smoke test for login handler

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-27 15:00:05 -04:00
Marc-Aurèle Brothier
bded5e98ec API: update url field max length 2016-05-27 08:38:24 +02:00
Marc-Aurèle Brothier
d0453fc296 not needed on host table 2016-05-27 08:26:09 +02:00
Marc-Aurèle Brothier
a59ee03fd7 Fix URL length to 2048 for all url fields in VO 2016-05-27 08:16:05 +02:00
Will Stevens
ecac28ba4e Merge pull request #1561 from sanju1010/startport
CLOUDSTACK-9388: Remove string conversion in assertion statementRemove string convertion in Assertion statement, since the start port parameter in listFirewallAPI response is of type integer

Test Result:
=========
"Checking firewall rules deletion after static NAT disable ... === TestName: test_01_firewall_rules_port_fw | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 1 test in 153.974s

OK

* pr/1561:
  CLOUDSTACK-9388: Remove string conversion in assertion statement

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-26 15:53:01 -04:00
Will Stevens
ff0b095028 Merge release branch 4.8 to master
* 4.8:
  lower the time we wait for interfaces to appear
2016-05-26 15:51:31 -04:00
Will Stevens
7974bfd313 Merge release branch 4.7 to 4.8
* 4.7:
  lower the time we wait for interfaces to appear
2016-05-26 15:50:29 -04:00
Will Stevens
d9429f6add Merge pull request #1471 from remibergsma/47_lower_interface_wait
Lower the time we wait for interfaces to appearWaiting for interfaces is tricky. They might never appear.. for example when we have entries in `/etc/cloudstack/ips.json` that haven't been plugged yet. Waiting this long makes everything horribly slow (every vm, interface, static route, etc, etc, will hit this wait, for every device). We've seen CloudStack send an `ip_assoc.json` command for `eth1` public nic only and then the router goes crazy waiting for all other interfaces that were there before reboot and aren't there. If only the router would return to the mgt server a success of `eth1`, it would get the command for `eth2` etc etc. Obviously, a destroy works much faster because no state services, so no knowledge of previous devices so no waits :-)

After a stop/start the router has state in `/etc/cloudstack/ips.json` and every commands waits. Eventually hitting the hardcoded 120 sec timeout.

* pr/1471:
  lower the time we wait for interfaces to appear

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-26 15:49:50 -04:00
Syed
26227768a1 [CLOUDSTACK-9296] Start ipsec for client VPN 2016-05-26 11:25:09 -04:00
jeff
9c1c50e05b Add lsb-release dependency to mgmt server and agent on Debian/Ubuntu. 2016-05-26 11:25:22 +00:00
jeff
7aec943ef8 Emit template UUID and class type over event bus when deleting templates.
The behavior is now consistent with template creation. This commit
also adds a unit test for this functionality to make sure that it will
always happen.
2016-05-26 11:20:39 +00:00
Will Stevens
3c800b4bfe Merge pull request #1518 from nvazquez/testnfs
CLOUDSTACK-9368: Fix for Support configurable NFS version for Secondary Storage mounts## Description
JIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9368
This pull request address a problem introduced in #1361 in which NFS version couldn't be changed after hosts resources were configured on startup (for hosts using `VmwareResource`), and as host parameters didn't include `nfs.version` key, it was set `null`.

## Proposed solution
In this proposed solution `nfsVersion` would be passed in `NfsTO` through `CopyCommand` to `VmwareResource`, who will check if NFS version is still configured or not. If not, it will use the one sent in the command and will set it to its storage processor and storage handler. After those setups, it will proceed executing command.

* pr/1518:
  CLOUDSTACK-9368: Fix for Support configurable NFS version for Secondary Storage mounts

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:05:07 -04:00
Will Stevens
978184bccb Merge pull request #1424 from syed/create-template-api-bug
[CLOUDSTACK-8973] Fix create template from snapshot returning null in case of region storeThis PR fixes a case where when we create a template from a snpashot in a region wide store, we don't set the cross zone flag which causes a null response to be returned
Tests:

Before fix
```
(local)  > create template snapshotid=33aa3f3b-5a47-4d2a-8d27-12952c01ebed displaytext=t2 ostypeid=20c8ead6-d750-11e5-9f8c-06524200007c name=t9

accountid = 1b13d7c2-d750-11e5-9f8c-06524200007c
cmd = org.apache.cloudstack.api.command.admin.template.CreateTemplateCmdByAdmin
created = 2016-02-23T16:09:24+0000
jobid = 4f9f5ff9-e7f0-4af6-999c-799431fd47de
jobinstanceid = a08a9711-bd31-43bb-80a2-49cf9d722a19
jobinstancetype = Template
jobprocstatus = 0
jobresult:
null:
crossZones = False
isfeatured = False
ispublic = False
isready = False
tags:
jobresultcode = 0
jobresulttype = object
jobstatus = 1
userid = 1b140f08-d750-11e5-9f8c-06524200007c
```

See the *null* in response

After fix:

```
(local)  > create template snapshotid=33aa3f3b-5a47-4d2a-8d27-12952c01ebed displaytext=t2 ostypeid=20c8ead6-d750-11e5-9f8c-06524200007c name=t11

accountid = 1b13d7c2-d750-11e5-9f8c-06524200007c
cmd = org.apache.cloudstack.api.command.admin.template.CreateTemplateCmdByAdmin
created = 2016-02-25T21:47:03+0000
jobid = 1b74209b-b3c1-4168-a243-f559aa0c081b
jobinstanceid = 06ecee5a-b1f2-4e67-80fb-f0f44b0aa198
jobinstancetype = Template
jobprocstatus = 0
jobresult:
template:
id = 06ecee5a-b1f2-4e67-80fb-f0f44b0aa198
name = t11
account = admin
created = 2016-02-25T21:47:03+0000
crossZones = True
displaytext = t2
domain = ROOT
domainid = 1b13ab80-d750-11e5-9f8c-06524200007c
format = VHD
hypervisor = XenServer
isdynamicallyscalable = False
isextractable = True
isfeatured = False
ispublic = False
isready = True
ostypeid = 20c8ead6-d750-11e5-9f8c-06524200007c
ostypename = CentOS 5 (64-bit)
passwordenabled = False
size = 21474836480
sourcetemplateid = 1af0f0cc-d750-11e5-9f8c-06524200007c
sshkeyenabled = False
status = Download Complete
tags:
templatetype = USER
jobresultcode = 0
jobresulttype = object
jobstatus = 1
userid = 1b140f08-d750-11e5-9f8c-06524200007c
```
Works correctly

* pr/1424:
  Fix create template from snapshot returning null in case of region store

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:04:31 -04:00
Will Stevens
6a6e164448 Merge pull request #1360 from greenqloud/pr-refactor-system-vm-network-creation
Refactor system VM default network creationTwo small commits which moves the retrieval of the default network for the console proxy and the SSVM into a separate protected method. It's a small change that makes the code more readable/maintainable and also makes the class more suitable for overriding should one want to do this. It's forward-ported from our 4.2 branch.

No new tests since this should not change any functionality, and thus should be covered by the existing unit tests.

Now on the master branch (#1359 was on the wrong branch).

* pr/1360:
  Refactor ssvm default network retrieval.
  Refactor console proxy default network retrieval.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:03:32 -04:00
Will Stevens
c122f4c879 Merge pull request #1552 from romain-dartigues/feature_dhcp_ubuntu
add DHCP lease folders for UbuntuAdd `/var/lib/dhcp/*` to the search path for Ubuntu; tested under Ubuntu Precise, Trusty and Xenial.

* pr/1552:
  add DHCP lease folders for Ubuntu

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:01:53 -04:00
Will Stevens
08d04e6de5 Merge pull request #1556 from mike-tutkowski/change-var-names-back
Hyper-V communication broken by change in variable namesChange variable names back to fix communication with Hyper-V Agent:

https://issues.apache.org/jira/browse/CLOUDSTACK-9383

* pr/1556:
  Removing “_” from the front of member variables in “Command” classes

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:01:22 -04:00
Will Stevens
b5606dcce8 Merge pull request #1553 from nlivens/mysql_driver_issue
Dynamically load drivers before creating our DB connectionsSolution to the mailing thread titled "MySQL : No suitable driver found for jdbc:mysql".
It doesn't harm that we explicitely load the MySQL driver, and for those which would use a commons-dbcp version < 1.4 this would fix it as well. Since JDBC 4.0, the JDBC driver can auto-register itself, but for some weird cases (like ours), it's not working. Therefore we need to explicitly load the JDBC driver.

* pr/1553:
  Dynamic loading of DB driver + support for other DB providers

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:00:42 -04:00
Will Stevens
de205c5805 Merge pull request #1297 from DaanHoogland/CLOUDSTACK-9203
CLOUDSTACK-9203 Implement security group move on updateVM API call  cherry-picked from a exoscale internal fix

Conflicts:
	api/src/org/apache/cloudstack/api/command/user/vm/UpdateVMCmd.java
	server/src/com/cloud/vm/UserVmManager.java
	server/src/com/cloud/vm/UserVmManagerImpl.java

* pr/1297:
  CLOUDSTACK-9203 refactorred DeployVM code to be used by UpdateVM as well
  CLOUDSTACK-9203 security group update on running instance

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 22:59:35 -04:00
Will Stevens
550aabb308 Merge pull request #1550 from ustcweizhou/CLOUDSTACK-9380-listDomains-NPE
CLOUDSTACK-9380: fix NPE in listDomains API for a mistakeThe issue happens if volumeTotal is NULL in database.
This is caused by commit 0407fb334f3a79f570217f35636b47076b06d500 for CLOUDSTACK-7847.

* pr/1550:
  CLOUDSTACK-9380: fix NPE in listDomains API for a mistake

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 22:58:46 -04:00
Will Stevens
05a7beccfe Merge pull request #1494 from nlivens/master_nuage
Remodeling of Nuage VSP Plugin + CLOUDSTACK-9294Hi all,

We've remodeled the Nuage VSP plugin to use the same model as VMWare is using (non-OSS). Before, we had a runtime dependency to the Nuage Client, this has been changed to a compile-time dependency instead because of multiple reasons (build management, readability, maintainability, ...)

We've adapted the code so it now uses model objects defined in the Nuage client instead of passing a list of parameters to the Nuage client. This is a lot more readable, and a lot more maintainable.

I've had a chat with @DaanHoogland about this approach, and he told me that ACS is trying to move away from the whole non-OSS approach. We're looking into the Juniper approach, we would set up a custom maven repository which would host the required dependencies for the Nuage VSP plugin.

Any remarks or suggestions are always welcome :)

* pr/1494:
  Nuage VSP : Extending Marvin test coverage
  Nuage VSP : Fix for NPE while cleaning up account when there are still resources belonging to that account
  CLOUDSTACK-9294 : Make sure to remove VR from VSD when removing the VPC
  CLOUDSTACK-9242 : Remodel Nuage VSP plugin

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 22:58:10 -04:00
Will Stevens
2d825dd07f Merge pull request #1555 from milamberspace/Improve-CS-1475
Add Java Default Certificat Authorities into the keystore if using a custom cert SSL

Related to CLOUDSTACK-1475

* pr/1555:
  Add Java Default Certificat Authorities into the keystore if using a custom cert SSL Related to CLOUDSTACK-1475 Fix some english message

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 22:57:40 -04:00
Will Stevens
b03a629c6a Merge pull request #1533 from greenqloud/pr-patchviasocket-convert-to-python
Convert patchviasocket to python (removes perl dependency for KVM agent)As requested here: https://github.com/apache/cloudstack/pull/1495

No scripts are using perl so that install requirement can be removed.
The new scripts are using standard python packages only.
Includes extensive unit test.
Note: perl-modules requirement is missing (fixed in mentioned PR) so do not merge that onto master.

* pr/1533:
  Revert "Add perl-modules as install dependency for cloudstack-agent"
  patchviasocket improve error handling
  Convert patchviasocket to python (removes perl dependency for KVM agent)

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 22:57:08 -04:00