100 Commits

Author SHA1 Message Date
Rohit Yadav
212e5ccfa7 CLOUDSTACK-9782: Host HA and KVM HA provider
Host-HA offers investigation, fencing and recovery mechanisms for host that for
any reason are malfunctioning. It uses Activity and Health checks to determine
current host state based on which it may degrade a host or try to recover it. On
failing to recover it, it may try to fence the host.

The core feature is implemented in a hypervisor agnostic way, with two separate
implementations of the driver/provider for Simulator and KVM hypervisors. The
framework also allows for implementation of other hypervisor specific provider
implementation in future.

The Host-HA provider implementation for KVM hypervisor uses the out-of-band
management sub-system to issue IPMI calls to reset (recover) or poweroff (fence)
a host.

The Host-HA provider implementation for Simulator provides a means of testing
and validating the core framework implementation.

Signed-off-by: Abhinandan Prateek <abhinandan.prateek@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-08-30 18:06:48 +02:00
Wido den Hollander
84e496b4f9
CLOUDSTACK-676: IPv6 Basic Security Grouping for KVM
This commit implements basic Security Grouping for KVM in
Basic Networking.

It does not implement full Security Grouping yet, but it does:
- Prevent IP-Address source spoofing
- Allow DHCPv6 clients, but disallow DHCPv6 servers
- Disallow Instances to send out Router Advertisements

The Security Grouping allows ICMPv6 packets as described by RFC4890
as they are essential for IPv6 connectivity.

Following RFC4890 it allows:
- Router Solicitations
- Router Advertisements (incoming only)
- Neighbor Advertisements
- Neighbor Solicitations
- Packet Too Big
- Time Exceeded
- Destination Unreachable
- Parameter Problem
- Echo Request

ICMPv6 is a essential part of IPv6, without it connectivity will break or be very
unreliable.

For now it allows any UDP and TCP packet to be send in to the Instance which
effectively opens up the firewall completely.

Future commits will implement Security Grouping further which allows controlling UDP and TCP
ports for IPv6 like can be done with IPv4.

Regardless of the egress filtering (which can't be done yet) it will always allow outbound DNS
to port 53 over UDP or TCP.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2017-01-26 15:36:08 +01:00
Rohit Yadav
6bdc411ff2
Merge branch '4.9' 2016-12-08 00:04:26 +05:30
Murali Reddy
6749785cab CLOUDSTACK-9339 Virtual Routers don't handle Multiple Public Interfaces correctly
-when processing static nat rule, add a mangle table rule, to mark the traffic
   from the guest vm when it has associated static nat rule so that traffic gets
   routed using the route tabe of the device which has public ip associated

  -fix the case where nic_device_id is empty when ip is getting disassociated
   resulting in empty deviceid in ips.json

  -add utility methods in CsRule, and CsRoute to add 'ip rule' and 'ip route' rules respectivley

  -ensure traffic from all public interfaces are connection marked with device number, and restored
   for the reverse traffic. use the connection marked number to do device specific routing table lookup
   fill the device specific routing table with default route

  -component tests for testing multiple public interfaces of VR
2016-12-07 14:33:24 +05:30
Rohit Yadav
0642a6982f
Merge branch '4.9' 2016-11-23 14:22:15 +05:30
Rohit Yadav
55b918076f
Merge branch '4.8' into 4.9 2016-11-23 13:50:15 +05:30
Rohit Yadav
ff616e700b Merge pull request #1745 from shapeblue/CLOUDSTACK-9503
CLOUDSTACK-9503: Increased the VR script timeout. Most of the changes are about converting int/long time values to joda Duration.

* pr/1745:
  CLOUDSTACK-9503: Increased the VR script timeout. Most of the changes are about converting int/long time values to joda Duration.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-23 13:41:52 +05:30
Abhinandan Prateek
83b5a8b2b2 CLOUDSTACK-9503: Increased the VR script timeout. Most of the changes are about converting int/long time values to joda Duration. 2016-11-01 16:14:23 +05:30
Mike Tutkowski
b508fb8692 Adding support for cross-cluster storage migration for managed storage when using XenServer 2016-09-12 07:39:13 -06:00
Will Stevens
1f9bf93948 Merge pull request #1598 from syed/vhd-compressed-size
[CLOUDSTACK-9423] Add ability to get virtual size of compressed VHDsWith object store like Swift as secondary storage, if a compressed VHD is uploaded as a template, the `VHDProcessor` incorrectly calculates the virutal size leading to the template being useless. This fix tries to guess the virtual size by partially decompressing it and falls back to a sensible default which is the size of the file.

Before the fix: template.properties on Swift
```
uniquename=routing-1
filename=routing-1.vhd
size=263417314
virtualsize=2894447637315205059
```
After the fix

```
uniquename=routing-1
filename=routing-1.vhd
size=263417314
virtualsize=3145728000
```

Look at the `virutalsize` in both cases

* pr/1598:
  [CLOUDSTACK-9423] Add ability to get virtual size of compressed VHDs

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-03 23:34:56 -04:00
Syed
b0247b53f9 [CLOUDSTACK-9423] Add ability to get virtual size of compressed VHDs 2016-06-29 12:53:51 -04:00
Will Stevens
f7f23ec720 Merge release branch 4.8 to master
* 4.8:
  CLOUDSTACK-9353: [XenServer] Fixed VM migration with storage
  Added ASF license to unit test file
  Added unit test to verify ordering
  Fixed ordering of network ACL rules being sent to the VR. The comparator was inverted
2016-06-28 11:21:04 -04:00
Patrick Dube
9cdd23fdc7 Added ASF license to unit test file 2016-06-03 08:48:47 -04:00
Patrick Dube
4c97a3981d Added unit test to verify ordering 2016-06-02 13:44:39 -04:00
Koushik Das
d1def0a730 Merge pull request #1287 from DaanHoogland/securityrules-cleanup
SecurityGroupRulesCmd code cleanupWrote a test and cleaned some duplicate code with the objective to evaluate the jenkins pull request process at builds.a.o
worthwhile to keep, IMHO.

* pr/1287:
  SecurityGroupRulesCmd code cleanup review comments handled
  deal with PMD warnings
  code cleanup
  security rules test
  remove autogenerated pydev files

Signed-off-by: Koushik Das <koushik@apache.org>
2016-04-11 21:49:57 +05:30
Daan Hoogland
b9b5967d6b SecurityGroupRulesCmd code cleanup review comments handled 2016-01-17 16:24:54 +01:00
Michael Andersen
4a08dbe235 [TEST] unittest needs rework 2016-01-07 19:27:42 +01:00
Daan Hoogland
1ead444cca security rules test 2015-12-26 09:02:28 +01:00
Wilder Rodrigues
6477bd8ff7 CLOUDSTACK-9067 - Remove old script file from the project
- Java constants also removed
   - Project still compiling and all unit tests passing.
2015-11-17 15:58:22 +01:00
Wido den Hollander
ea92fc15ce Use java.io.tmpdir instead of hardcoded /tmp
This was submitted earlier in PR #884 but that did not merge
properly.

This is a new PR with the same change.
2015-10-31 12:03:11 +01:00
Aaron Brady
cacac14425 Pass LbProtocol down to the HAProxyConfigurator 2015-10-28 13:42:51 +00:00
Rajani Karuturi
1056171aca CLOUDSTACK-8808: Successfully registered VHD template is downloaded
again due to missing virtualsize property in template.properties

We have multiple file processors to process different types of image
formats. The processor interface has two methods getVirtualSize() and
process().

    1. getVirtualSize() as the name says, returns the virtual size of
the file and is used at get the size while copying files from NFS to s3
    2. process() returns FormatInfo struct which has fileType, size,
virutalSize, filename.  on successfully downloading a template, each
file is passed to all the processors.process() and whichever returns a
FormatInfo, that will be used to create template.properties file.  If
process() throws an InternalErrorException, template installation fails.
But, if process() returns null, template registration is successful with
template.properties missing some attributes like virtualSize, file
format etc. which results in this bug on restart of ssvm/cloud
service/management server.

failing the template download if virutalsize or some other properties
cannot be determined.

The following changes are done:
getVirtualSize() to always return size(if it can calculate, get virtual
size else return file size). This would mean the following changes

    1. QCOW2Processor.getVirtualSize() to return file size if virtual
size calculation fails
    2. VHDProcessor.getVirtualSize() to return file size if virtual size
calculation fails

process() to throw InternalErrorException if virtual size calculation
fails or any other exceptions occur. This would mean the following
changes

    1. OVAProcessor to throw InternalErrorException if untar fails
    2. QCOW2Processor to throw InternalErrorException if virtual size
calculation fails
    3. VHDProcessor to throw InternalErrorException if virtual size
calculation fails
2015-09-30 15:43:51 +05:30
Remi Bergsma
415631ab58 Revert "Merge pull request #884 from wido/test-tmp-dir"
This reverts commit 6841ba61da5e407f7a16c4a575d1a4e8c8345970, reversing
changes made to 13b29bac5a1778e295df7e9fb21c502fcf017183.

Master is currently frozen, no merges without RM approval.
http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201509.mbox/browser

It also broke the build:
[INFO] Apache CloudStack Framework - Jobs ................ SUCCESS [3.448s]
[INFO] Apache CloudStack Cloud Engine Internal Components API  SUCCESS [2.528s]
[INFO] Apache CloudStack Server .......................... FAILURE [24.769s]
[INFO] Apache CloudStack Usage Server .................... SKIPPED
2015-09-25 11:41:03 +02:00
Wido den Hollander
9e90b5393a Use java.io.tmpdir instead of hardcoded /tmp 2015-09-24 15:07:41 +02:00
Boris Schrijver
c3f2ea6ae0 Added Unit Test for LocalTemplateDownloader. 2015-09-10 13:07:34 +02:00
Koushik Das
e1db6efc74 Removed unused agent command AttachVolumeCommand and corresponding answer AttachVolumeAnswer 2015-08-26 09:50:23 +05:30
wilderrodrigues
3d22a16c4f Bump priority stragety is no longer used for redundant virtual routers
- With the changes added by the rVPC work, the bump priority became deprecated.
     This commit includes a refactor to get it removed from the following resources:
     * Java classes
     * domain_router table - removing the is_priority_bumpup column
     * Fixing unit tests

All changes were tested with:

XenServer 6.2 running under our VMWare zone
CloudStack Management Server running on MacBook Pro
MySql running on MackBook Pro
Storage Type: Local
2015-04-02 21:50:49 +02:00
wilderrodrigues
57d0931b49 Adding more tests to cover other possibilities concerning different versions of the XenServer
Adding a better algorithm in order to corver deeper inheritance form the command classes
Removing some unsued variables.
2015-03-31 16:49:48 +02:00
Hugo Trippaers
39035f4782 Fix test cases now ConfigHelper is gone 2015-03-16 11:35:38 +01:00
wilderrodrigues
0a1677242a Adding tests for IpAssocVpc; DeleteIpAlias; DnsMasqConfig
Adding IpAssocVpcCommand to the AbstractConfigItemFacade map
2015-03-16 11:35:34 +01:00
wilderrodrigues
6d7f75e18b Adding LoadBalancer command to JSON style
Adding a couple of tests (ConfigHelperTest) to validate the following:

* Command is generated;
* JSON String is generated based on the Java Object
* JSON Object is generated based on the String content

Also fixed the SetPortForwardingRulesVpcConfigItem: implementation was missing.
2015-03-16 11:35:33 +01:00
Hugo Trippaers
bda4c0d2c9 Move some static strings to constants and remove some duplicate code 2015-03-16 11:35:11 +01:00
Hugo Trippaers
25b8510c43 Change vmdata to the new config system 2015-03-16 11:35:06 +01:00
Hugo Trippaers
1f1a9ba8b0 Create a json file for SetNetworkACL 2015-03-16 11:35:03 +01:00
Hugo Trippaers
639b24a575 Replace CreateGuestNetwork config script with a json file. 2015-03-16 11:35:01 +01:00
Hugo Trippaers
0f5af91668 Switch ip associations to the new config file system 2015-03-16 11:34:58 +01:00
Laszlo Hornyak
df11bb2811 CLOUDSTACK-8279: Fix unit test for java 1.8
Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
2015-02-27 20:06:27 +01:00
Laszlo Hornyak
f8c003d13a CLOUDSTACK-8279: test fix for java 1.8
The test build on an assumption on the order of items in a Set, while this changed in java 1.8

Signed-off-by: Laszlo Hornyak <laszlo.hornyak@gmail.com>
2015-02-27 19:38:12 +01:00
Fred Clift
355eb72c7d make virtual router /latest/.htaccess file be static - no longer dynamically generate it, remove code that managed the file previously
Signed-off-by: Sebastien Goasguen <runseb@gmail.com>
2014-09-04 04:22:27 -04:00
Jayapal
1bfb1f6503 CLOUDSTACK-7246: corrected script name of createipalias 2014-08-05 14:41:12 +05:30
Sheng Yang
717e697246 Revert "CLOUDSTACK-7124: Fix unit test"
This reverts commit ef28fd36727bfffbbcb7f2fc120139b7c7d39eb2.

Fix of CLOUDSTACK-7124 caused CLOUDSTACK-7163.
2014-07-25 14:08:20 -07:00
Sheng Yang
ef28fd3672 CLOUDSTACK-7124: Fix unit test 2014-07-17 19:40:45 -07:00
Hugo Trippaers
a49bb7e8e3 Cleanup licenses in core with com.mycila:license-maven-plugin:format 2014-07-14 16:28:29 +02:00
Hugo Trippaers
c12767908d Refactor VirtualRoutingResource, put the ConfigItems in separate classes and create a separate ConfigHelper 2014-07-14 15:32:31 +02:00
Edison Su
25a6234a5b fix build 2014-03-28 16:24:45 -07:00
Sheng Yang
1d3a6eaa50 CLOUDSTACK-6047: Fix timeout issue when try to execute aggregated commands
Add executeInVR() with timeout interface to VirtualRouterDeployer

AggregationControlCommand with Action.Finish may take longer than normal command
since it would execute all the commands in one execution, and it may result in
SSH timeout for SshHelper or other mechanism communicate with VR.

Introduce an new executeInVR() interface with added timeout period for waiting
FinishAggregationCommand to complete execution.
2014-03-18 17:43:59 -07:00
Sheng Yang
f45de30d1b CLOUDSTACK-6047: Enable VR aggregation commands for VR start/reboot 2014-03-11 13:04:26 -07:00
Sheng Yang
2de67dff42 CLOUDSTACK-6047: Fix checkstyle build issue 2014-03-06 16:49:07 -08:00
Sheng Yang
ccea5703df CLOUDSTACK-6047: Add testing for VR aggregation commands 2014-03-06 16:22:23 -08:00
Sheng Yang
c0ed0b7819 CLOUDSTACK-6047: Make VR LB script accept a file name
Rather than default filename everytime since in aggregated commands the file
would be overrided if the filename is same.
2014-02-21 18:26:42 -08:00