930 Commits

Author SHA1 Message Date
John Bampton
6401c850b7
Fix spelling (#6064)
* Fix spelling

- `interupted` to `interrupted`
- `paramter` to `parameter`

* Fix more typos
2022-03-08 13:02:35 -03:00
John Bampton
980c0bff03
Fix spelling (#6041)
Changed `occured` to `occurred`
2022-03-03 13:10:14 -03:00
Gabriel Beims Bräscher
b4db3db617
Use default timeout and retransmission values for the NFS mount. (#6019)
This also allows the mount command to apply NFS mount custom values set by ADMINS via '/etc/nfsmount.conf'.
2022-03-02 09:07:08 -03:00
Pearl Dsilva
e0a5df50ce
CKS Enhancements and SystemVM template upgrade improvements (#5863)
* This PR/commit comprises of the following:
- Support to fallback on the older systemVM template in case of no change in template across ACS versions
- Update core user to cloud in CKS
- Display details of accessing CKS nodes in the UI - K8s Access tab
- Update systemvm template from debian 11 to debian 11.2
- Update letsencrypt cert
- Remove docker dependency as from ACS 4.16 onward k8s has deprecated support for docker - use containerd as container runtime

* support for private registry - containerd

* Enable updating template type (only) for system owned templates via UI

* edit indents

* Address comments and move cmd from patch file to cloud-init runcmd

* temporary change

* update k8s test to use k8s version 1.21.5 (instead of 1.21.3 - due to https://github.com/kubernetes/kubernetes/pull/104530)

* support for private registry - containerd

* Enable updating template type (only) for system owned templates via UI

* smooth upgrade of cks clusters

* update pom file with temp download.cloudstack.org testing links

* fix pom

* add cgroup config for containerd

* add systemd config for kubelet

* add additional info during image registry config

* update to official links
2022-02-15 18:27:14 +05:30
Pearl Dsilva
640118ce2b
Add disk space in systemVM template registration script (#5956) 2022-02-10 09:50:12 +05:30
Wei Zhou
6495bc1a47
packaging: display First Install and Onboarding Message (#5851)
* packaging: display First Install and Onboarding Message

* Update #5851: Update as per Rohit's comments

* Update #5851: display package name in help message

* Update #5851: display links of installed cloudstack version on UI

* Update #5851: fix vue warnings
2022-02-08 16:11:03 +01:00
Nicolas Vazquez
3e92a63155
[XenServer/XCP-ng] Pass the image store NFS version on storage commands (#5886)
* Add NFS version to mount command

* Remove extra line

* Extend NFS version to mount secondary storage

* Unused import

* Refactor NFS version to be granular

* Make use of the ConfigKey on the NFS version setting value
2022-01-31 12:21:13 +05:30
Wei Zhou
6b757d502e
CKS: use cluster-autoscaler-standard.yaml in kubernetes repo (#5574) 2021-10-13 12:46:52 -03:00
davidjumani
6ac834a358
Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements (#4329)
Adding AutoScaling support for cks
Kubernetes PR : kubernetes/autoscaler#3629
Also replaces CoreOS with Debian
Fixes #4198

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Wei Zhou <w.zhou@global.leaseweb.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-10-06 21:17:41 +05:30
Wei Zhou
09fce75047
OVS/GRE: bug fixes (#5446)
* OVS/GRE: fix failed to fetch cloudbr* info on host

* OVS/GRE: move ovstunnel.py to python3

* OVS: add test_privategw_acl_ovs_gre.py for automatic testing

(1) VR <-> VM should work
(2) Private GW should work

* OVS: fix an issue on xenserver

ovs bridges are deleted by xenserver/ovs automatically
```
[root@ref-trl-1797-x-M7-wei-zhou-xs2 ~]# grep -r xapi7 /var/log/ |grep del-br
/var/log/xensource.log:Sep 15 07:13:44 ref-trl-1797-x-M7-wei-zhou-xs2 xcp-networkd: [ info|localhost|611 |org.xen.xapi.xenops.classic events D:4a3d931cd89f|network_utils] /usr/bin/ovs-vsctl --timeout=20 -- --if-exists del-br xapi7
/var/log/daemon.log:Sep 15 07:13:45 ref-trl-1797-x-M7-wei-zhou-xs2 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=20 -- --if-exists del-br xapi7
```

which results that xe network exists but bridge does not exist, and operation stuck for 20 minutes at
```
2021-09-15 16:06:56    DEBUG [root] #### VMOPS enter  create_tunnel ####
2021-09-15 16:06:56    DEBUG [root] Creating tunnel from host 2 to host 1 with GRE key 2116
2021-09-15 16:06:56    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', '--timeout=0', 'wait-until', 'bridge', 'xapi7', '--', 'get', 'bridge', 'xapi7', 'name']
2021-09-15 16:26:56    DEBUG [root] bridge xapi7 for creating tunnel - VERIFIED
2021-09-15 16:26:56    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', 'add-port', 'xapi7', 't2116-2-1', '--', 'set', 'interface', 't2116-2-1', 'type=gre', 'options:key=2116', 'options:remote_ip=10.0.34.230']
```

* OVS/GRE: component test test_vpc_distributed_routing_offering.py passes on xenserver

* OVS: support Shared network with GRE protocol

* update #5446: use lang3.StringUtils

* OVS/GRE: VPC with distributed routing on kvm (still not working)
2021-10-03 14:47:52 +05:30
davidjumani
8bc120b288
Merge remote-tracking branch 'apache/4.15' into main 2021-08-24 13:46:28 +05:30
davidjumani
f822547c3b
securitygroups: Fix security_groups for c8/suse (#5354)
* Fix security_groups for c8/suse

* security_group.py: revert a minor change in #5348

Co-authored-by: Wei Zhou <weizhou@apache.org>
2021-08-24 13:07:52 +05:30
Rohit Yadav
57a55bd777 Merge remote-tracking branch 'origin/4.15' into main 2021-08-20 17:19:40 +05:30
Wei Zhou
f9fe9da734
security group: fix component test test_multiple_nic_support.py failures (#5348)
This PR fixes component test failure with test_multiple_nic_support.py, in advanced zone with security groups.

Co-authored-by: dahn <daan.hoogland@gmail.com>
2021-08-20 17:06:26 +05:30
Rohit Yadav
d841709efc Merge remote-tracking branch 'origin/4.15' into main 2021-08-20 16:52:32 +05:30
Rakesh
c1fc002359
Fix iptable rules when chain reference count is 0 (#5328)
Sometimes the chain reference count is 0 and in that case
proper iptables rules are not applied. Because of this, ping fails.
So check the reference count for the main chain and as well as
-IN and -OUT chain as well

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-08-20 13:17:45 +02:00
davidjumani
f17683779c
packaging: Adding SUSE 15 support (#5110)
This PR adds support for SUSE Leap 15.2+ as a KVM hypervisor and Management / Usage server on Cloudstack
Related docs link apache/cloudstack-documentation#224

Co-authored-by: Michael <35783820+mib1185@users.noreply.github.com>
2021-08-19 13:28:54 +05:30
Rakesh
5ed3246e41
Fix iptable rules in ubuntu 20 for bridge name (#5318)
In ubuntu20 the interface name contains @ synbol and
because of that even the iptable rules for brdige name
contains this symbol which causes ping issues.
Remove the @ symbol from iptable rule to fix the issue

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-08-19 13:15:26 +05:30
davidjumani
98d3231dbf
Adding support for RHEL8 binary-compatible variants (#5158)
* Adding support for RHEL binary-compatible variants

* Skip ipmi related tests

* Fixing security_groups.py
2021-08-18 10:03:03 +02:00
davidjumani
3c8c704df3
kubernetes: Deploy kubernetes-provider when creating a cluster (#5254)
* kubernetes: Deploy kubernetes-provider when creating a cluster

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-08-09 14:45:20 +05:30
Daniel Augusto Veronezi Salvador
82df04ecc8
Improve HA logs (#5241)
Co-authored-by: GutoVeronezi <daniel@scclouds.com.br>
2021-07-30 21:13:16 +02:00
Abhishek Kumar
5ac184edd6 Merge remote-tracking branch 'apache/4.15' into main 2021-07-15 14:02:49 +05:30
Wei Zhou
cf0f1feb5e
configdrive: fix some failures in tests/component/test_configdrive.py (#5144)
* server: fix failed to apply userdata when enable static nat

* server: fix cannot expunge vm as applyUserdata fails

* configdrive: fix ISO is not recognized when plug a new nic

* configdrive: detach and attach configdrive ISO as it is changed when plug a new nic or migrate vm

* configdrive test: (1) password file does not exists in recreated ISO; (2) vm hostname should be changed after migration

* configdrive: use centos55 template with sshkey and configdrive support

* configdrive: disklabel is 'config-2' for configdrive ISO

* configdrive: use copy for configdrive ISO and move for other template/volume/iso

* configdrive: use public-keys.txt

* configdrive test: fix (1) update_template ; (2) ssh into vm by keypair
2021-07-15 12:49:37 +05:30
Daniel Augusto Veronezi Salvador
99f2919ef4
Improve logs on kvmvmactivity.sh (#4704)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-05-10 16:26:55 +02:00
Pearl Dsilva
0dbeb262e4
server: Support for persistence mode in L2 networks (#4561)
This PR aims at introducing persistence mode in L2 networks and enhancing the behavior in Isolated networks
Doc PR apache/cloudstack-documentation#183

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-05 14:37:11 +05:30
Daniel Augusto Veronezi Salvador
4e90a8c454
Qemu 2.10 requires -U flag to read volume metadata (#4567)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-03-18 11:54:01 +01:00
Abhishek Kumar
e13363ed76
cks: fix CNI release url returning 404 (#4684)
CNI plugin release naming has changed, https://github.com/containernetworking/plugins/releases
Release are named for host OS from 0.8.0 onwards.
This change adds check for 404 response code and attempts retry

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-23 16:26:17 +05:30
Daan Hoogland
4fd2b74f57 Merge branch '4.14' 2020-11-20 08:59:51 +00:00
davidjumani
1398dfff65
createkubertetesbinariesiso: Saving images in network and dashboard yaml (#4459) 2020-11-20 13:49:26 +05:30
Rohit Yadav
d6db47618d Merge remote-tracking branch 'origin/4.14' 2020-10-14 16:06:57 +05:30
Pearl Dsilva
cdcc914995
scripts: creates /var/run/cloud folder for creation of lock file (#4373)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-10-14 15:45:22 +05:30
Rohit Yadav
b3bafffff3 Merge remote-tracking branch 'origin/4.14' 2020-09-29 14:33:58 +05:30
Abhishek Kumar
9391fa9b6b
cks: assorted fixes, test refactoring (#4269)
Fixes #4265
Fixes for:

template selection
network cleanup
capacity check
specific patch version during deployment

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-09-29 14:31:41 +05:30
Wei Zhou
00ceafe47f
securitygroup: Ubuntu 20.04 fix systemvm cannot start up (#4303)
* security_group.py: fix SyntaxWarning: "is" with a literal.

2020-04-27 09:43:54,172 DEBUG [kvm.resource.LibvirtComputingResource] (Agent-Handler-2:null) (logid:c33ba330) /usr/share/cloudstack-common/scripts/vm/network/security_group.py:513: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if rules is None or rules is "":
/usr/share/cloudstack-common/scripts/vm/network/security_group.py:522: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if rules is None or rules is "":
/usr/share/cloudstack-common/scripts/vm/network/security_group.py:823: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if brName is None or brName is "":

* Ubuntu 20.04: Fix systemvm cannot start up

in Ubuntu 16.04:

root@node13:~# bridge -o link show
2: eth0 state UP : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100
5: vnet0 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloud0 state forwarding priority 32 cost 100
6: vnet1 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100
7: vnet2 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100

root@node13:~# bridge -o link show | awk '/master cloudbr0 / && !/^[0-9]+: vnet/ {print $2}' | head -1
eth0

root@node13:~# bridge -o link show | awk '/master cloudbr0 / && !/^[0-9]+: vnet/ {print $2}' | head -1 |cut -d ":" -f1
eth0

in Ubuntu 20.04:

root@node62:~# bridge -o link show
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100
10: vnet3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloud0 state forwarding priority 32 cost 100
11: vnet4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100
12: vnet5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100

root@node62:~# bridge -o link show | awk '/master cloudbr0 / && !/^[0-9]+: vnet/ {print $2}' | head -1
ens3:

root@node62:~# bridge -o link show | awk '/master cloudbr0 / && !/^[0-9]+: vnet/ {print $2}' | head -1 |cut -d ':' -f1
ens3

* security_group.py: use 'if not' instead
2020-09-22 20:56:54 +05:30
davidjumani
3872bf1ff9
kvm: Enable PVLAN support on L2 networks (#4040)
This is an extention of #3732 for kvm.
This is restricted to ovs > 2.9.2
Since Xen uses ovs 2.6, pvlan is unsupported.
This also fixes issues of vms on the same pvlan unable to communicate if they're on the same host
2020-08-20 15:46:34 +05:30
davidjumani
d949302d0f
packaging: Adding Centos8, Ubuntu 20.04, XCPNG8.1 Support (#4068)
* DB : Add support for MySQL 8

- Splits commands to create user and grant access on database, the old
statement is no longer supported by MySQL 8.x
- `NO_AUTO_CREATE_USER` is no longer supported by MySQL 8.x so remove
that from db.properties conn parameters

For mysql-server 8.x setup the following changes were added/tested to
make it work with CloudStack in /etc/mysql/mysql.conf.d/mysqld.cnf and
then restart the mysql-server process:

    server_id = 1
    sql-mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION"
    innodb_rollback_on_timeout=1
    innodb_lock_wait_timeout=600
    max_connections=1000
    log-bin=mysql-bin
    binlog-format = 'ROW'

    default-authentication-plugin=mysql_native_password

Notice the last line above, this is to reset the old password based
authentication used by MySQL 5.x.

Developers can set empty password as follows:

    > sudo mysql -u root
    ALTER USER 'root'@'localhost' IDENTIFIED BY '';

In libvirt repository, there are two related commits

2019-08-23 13:13 Daniel P. Berrangé            ● rpm: don't enable socket activation in upgrade if --listen present
2019-08-22 14:52 Daniel P. Berrangé            ● remote: forbid the --listen arg when systemd socket activation

In libvirt.spec.in

        /bin/systemctl mask libvirtd.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-ro.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-admin.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tls.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tcp.socket >/dev/null 2>&1 || :

Co-authored-by: Wei Zhou <w.zhou@global.leaseweb.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-17 16:28:30 +05:30
ServiceTo
e02801bf00
xenserver: add support for XCP-ng 7/8 to create it's heartbeat LVM properly (#4109)
/opt/cloud/bin/setup_heartbeat_sr.sh line 75 tests for “XenServer release 7.” in /etc/redhat-release which contains “XCP-ng release 8.0.0 (xenenterprise)” in my case so the lvcreate statement doesn’t contain the required options

Fixes: #3281

Co-authored-by: Ian Service <ian@service.to>
2020-06-11 17:30:16 +05:30
Rohit Yadav
debde60c94 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-06-04 08:22:58 +05:30
Spaceman1984
23fa647985
kvm: sending std output to dev/null to prevent garbage output (#4123)
When scripts/vm/hypervisor/kvm/kvmvmactivity.sh is called with an incorrect file name, an error is printed which is then interpreted as output from the script.

When an incorrect file name is passed the script prints out:

stat: cannot stat ‘b51d7336-d964-44ee-be60-bf62783dabc’: No such file or directory
=====> DEAD <======
The KVMHAVMActivityChecker.java checkingHB() process is expecting just
=====> DEAD <======
but gets the unexpected error message and interprets the file as alive.
2020-06-04 08:17:59 +05:30
Abhishek Kumar
8cc70c7d87
CloudStack Kubernetes Service (#3680) 2020-03-06 08:51:23 +01:00
Wei Zhou
458d3b5b47
Multiple networks support for vms in advanced zone with securit… (#3639) 2020-02-19 14:02:12 +00:00
Rohit Yadav
d90341ebf1
cloudstack: add JDK11 support (#3601)
This adds support for JDK11 in CloudStack 4.14+:

- Fixes code to build against JDK11
- Bump to Debian 9 systemvmtemplate with openjdk-11
- Fix Travis to run smoketests against openjdk-11
- Use maven provided jdk11 compatible mysql-connector-java
- Remove old agent init.d scripts

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-02-12 12:58:25 +05:30
Abhishek Kumar
0f5b0e67f8
VM ingestion (#3606)
The VM ingestion feature allows CloudStack to discover, on-board, import existing VMs in an infra. The feature currently works only for VMware, with a hypervisor agnostic framework which may be extended for KVM and XenServer in future.
2020-02-03 15:43:52 +01:00
Andrija Panic
9b7acfde1e
Update message when keys are NOT being injected (#3799)
* Update message when keys are NOT being injected

* Correct the message after injectkeys.ssh is done

* Update message to a more meaningful one, since sometimes nothing is injected

* Update other 2

* typo
2020-01-18 09:11:42 +01:00
Paul Angus
be97470d83 Get Diagnostics: Download logs and diagnostics data from SSVM, CPVM, Router (#3350)
* * Complete API implementation
* Complete UI integration
* Complete marvin test
* Complete Secondary storage GC background task

* improve UI labels

* slight reword and add another missing description

* improve download message clarity

* Address comments

* multiple fixes and cleanups

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix more bugs, let it return ip rule list in another log file

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix missing iprule bug

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* add support for ARCHIVE type of object to be linked/setup on secstorage

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Fix retrieving files for Xenserver

* Update get_diagnostics_files.py

* Fix bug where executable scripts weren't handled

* Fixed error on script cmd generation

* Do not filter name for log files as it would override similar prefix script names

* Addressed code review comments

* log error instead of printstacktrace

* Treat script as executable and shell script

* Check missing script name case and write to output instead of catching exception

* Use shell = true instead of shlex to support any executable

* fix xenserver bug

* don't set dir permission for vmware

* Code review comments - refactoring

* Add check for possible NPE

* Remove unused imoprt after rebase

* Add better description for configs

Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com>
Co-authored-by: Rohit Yadav <rohit@apache.org>
Co-authored-by: Anurag Awasthi <anurag.awasthi@shapeblue.com>
2020-01-15 11:38:33 +01:00
Anurag Awasthi
1771727ce1 Revert "Extract systemvm.iso using bsdtar (#3536)" (#3800)
This reverts commit 8a55c931e806a64f7f00153d4a58c4463caa7a47.
2020-01-13 12:24:43 +01:00
Rakesh
dfd950cab0 Add protocol number support for security group rules (#3736)
Currently while creating ingress/egress rule for a security group,
we can specify only TCP/UDP/ICMP. Sometimes we need to add rules
for different protocol number or rules for all the above three
mentioned protocols.

In this new feature users can specify the protocol number or select
"ALL" option which will apply rules for TCP/UDP/ICMP
2020-01-08 16:13:41 +01:00
Kai Takahashi
8a55c931e8 Extract systemvm.iso using bsdtar (#3536)
* Extract systemvm.iso using bsdtar if available.

Signed-off-by: Kai Takahashi <k-takahashi@creationline.com>

* New dependency for CentOS 7 and Debian: bsdtar

bsdtar can extract iso images without mounting.

Signed-off-by: Kai Takahashi <k-takahashi@creationline.com>

* Remove all 'mount' and 'umount' command call(s).

Signed-off-by: Kai Takahashi <k-takahashi@creationline.com>
2020-01-06 11:46:20 +01:00
Andrija Panic
45503e104a
Revert "Revert "Fix virtual template size for managed storage for KVM / refactor cloud-install-sys-tmplt (#3371)" (#3771)" (#3772)
This reverts commit e319c8b8f0e742b4f6139603d7881974d0a02e8f.
2019-12-19 19:50:30 +01:00
Andrija Panic
e319c8b8f0
Revert "Fix virtual template size for managed storage for KVM / refactor cloud-install-sys-tmplt (#3371)" (#3771)
This reverts commit a0efbf9d09e57a907cd599d28a1def02686240f1.
2019-12-17 15:29:37 +01:00