gfs2 cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: NONE
Last updated: Sat Oct 26 22:32:30 2019
Last change: Sat Oct 26 22:19:04 2019 by root via cibadmin on cluster2

2 nodes configured
1 resource configured

Node cluster1: UNCLEAN (offline)
Node cluster2: UNCLEAN (offline)

Full list of resources:

 Resource Group: halvmfs
     halvm (ocf::heartbeat:LVM): Stopped

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# mkdir /xfs
[root@cluster1 ~]# clear

[root@cluster1 ~]# pcs resource create xfsfs filesystem device="/dev/vg/cluster" directory="/xfs"
Assumed agent name 'ocf:heartbeat:Filesystem' (deduced from 'filesystem')
Error: required resource option 'fstype' is missing, use --force to override
[root@cluster1 ~]# pcs status^C
[root@cluster1 ~]#  pcs resource create xfsfs filesystem device="/dev/vg/cluster" directory="/xfs" --force
Assumed agent name 'ocf:heartbeat:Filesystem' (deduced from 'filesystem')
Warning: required resource option 'fstype' is missing
[root@cluster1 ~]# pcs resource create xfsfs Filesystem device="/dev/vgcluster/lvcluster" directory="/xfs" fstype="xfs" --group halvmfs
Assumed agent name 'ocf:heartbeat:Filesystem' (deduced from 'Filesystem')
Error: 'xfsfs' already exists
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 22:40:45 2019
Last change: Sat Oct 26 22:37:03 2019 by root via cibadmin on cluster1

2 nodes configured
2 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Resource Group: halvmfs
     halvm (ocf::heartbeat:LVM): Stopped
 xfsfs (ocf::heartbeat:Filesystem): Stopped

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# mount | grep xfs
/dev/mapper/centos-root on / type xfs (rw,relatime,attr2,inode64,noquota)
/dev/sda1 on /boot type xfs (rw,relatime,attr2,inode64,noquota)
/dev/mapper/centos-root on /var/lib/docker/containers type xfs (rw,relatime,attr2,inode64,noquota)
/dev/mapper/centos-root on /var/lib/docker/overlay2 type xfs (rw,relatime,attr2,inode64,noquota)
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 22:51:20 2019
Last change: Sat Oct 26 22:42:10 2019 by root via cibadmin on cluster2

2 nodes configured
2 resources configured

Node cluster2: standby
Online: [ cluster1 ]

Full list of resources:

 Resource Group: halvmfs
     halvm (ocf::heartbeat:LVM): Stopped
 xfsfs (ocf::heartbeat:Filesystem): Stopped

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs resource start halvm

Usage: pcs resource [commands]...
Manage pacemaker resources

Commands:
    [show [<resource id>] | --full | --groups | --hide-inactive]
        Show all currently configured resources or if a resource is specified
        show the options for the configured resource.  If --full is specified,
        all configured resource options will be displayed.  If --groups is
        specified, only show groups (and their resources).  If --hide-inactive
        is specified, only show active resources.

    list [filter] [--nodesc]
        Show list of all available resource agents (if filter is provided then
        only resource agents matching the filter will be shown). If --nodesc is
        used then descriptions of resource agents are not printed.

    describe [<standard>:[<provider>:]]<type> [--full]
        Show options for the specified resource. If --full is specified, all
        options including advanced ones are shown.

    create <resource id> [<standard>:[<provider>:]]<type> [resource options]
           [op <operation action> <operation options> [<operation action>
           <operation options>]...] [meta <meta options>...]
           [clone [<clone options>] | master [<master options>] |
           --group <group id> [--before <resource id> | --after <resource id>]
           | bundle <bundle id>] [--disabled] [--no-default-ops] [--wait[=n]]
        Create specified resource. If clone is used a clone resource is
        created. If master is specified a master/slave resource is created.
        If --group is specified the resource is added to the group named. You
        can use --before or --after to specify the position of the added
        resource relatively to some resource already existing in the group.
        If bundle is used, the resource will be created inside of the specified
        bundle. If --disabled is specified the resource is not started
        automatically. If --no-default-ops is specified, only monitor
        operations are created for the resource and all other operations use
        default settings. If --wait is specified, pcs will wait up to 'n'
        seconds for the resource to start and then return 0 if the resource is
        started, or 1 if the resource has not yet started. If 'n' is not
        specified it defaults to 60 minutes.
        Example: Create a new resource called 'VirtualIP' with IP address
            192.168.0.99, netmask of 32, monitored everything 30 seconds,
            on eth2:
            pcs resource create VirtualIP ocf:heartbeat:IPaddr2 \
                ip=192.168.0.99 cidr_netmask=32 nic=eth2 \
                op monitor interval=30s

    delete <resource id|group id|master id|clone id>
        Deletes the resource, group, master or clone (and all resources within
        the group/master/clone).

    enable <resource id>... [--wait[=n]]
        Allow the cluster to start the resources. Depending on the rest of the
        configuration (constraints, options, failures, etc), the resources may
        remain stopped. If --wait is specified, pcs will wait up to 'n' seconds
        for the resources to start and then return 0 if the resources are
        started, or 1 if the resources have not yet started. If 'n' is not
        specified it defaults to 60 minutes.

    disable <resource id>... [--wait[=n]]
        Attempt to stop the resources if they are running and forbid the
        cluster from starting them again. Depending on the rest of the
        configuration (constraints, options, failures, etc), the resources may
        remain started. If --wait is specified, pcs will wait up to 'n' seconds
        for the resources to stop and then return 0 if the resources are
        stopped or 1 if the resources have not stopped. If 'n' is not specified
        it defaults to 60 minutes.

    restart <resource id> [node] [--wait=n]
        Restart the resource specified. If a node is specified and if the
        resource is a clone or master/slave it will be restarted only on
        the node specified.  If --wait is specified, then we will wait
        up to 'n' seconds for the resource to be restarted and return 0 if
        the restart was successful or 1 if it was not.

    debug-start <resource id> [--full]
        This command will force the specified resource to start on this node
        ignoring the cluster recommendations and print the output from
        starting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to start.

    debug-stop <resource id> [--full]
        This command will force the specified resource to stop on this node
        ignoring the cluster recommendations and print the output from
        stopping the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to stop.

    debug-promote <resource id> [--full]
        This command will force the specified resource to be promoted on this
        node ignoring the cluster recommendations and print the output from
        promoting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to promote.

    debug-demote <resource id> [--full]
        This command will force the specified resource to be demoted on this
        node ignoring the cluster recommendations and print the output from
        demoting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to demote.

    debug-monitor <resource id> [--full]
        This command will force the specified resource to be monitored on this
        node  ignoring the cluster recommendations and print the output from
        monitoring the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to be monitored.

    move <resource id> [destination node] [--master] [lifetime=<lifetime>]
         [--wait[=n]]
        Move the resource off the node it is currently running on by creating a
        -INFINITY location constraint to ban the node.  If destination node is
        specified the resource will be moved to that node by creating an
        INFINITY location constraint to prefer the destination node.  If
        --master is used the scope of the command is limited to the master role
        and you must use the master id (instead of the resource id).  If
        lifetime is specified then the constraint will expire after that time,
        otherwise it defaults to infinity and the constraint can be cleared
        manually with 'pcs resource clear' or 'pcs constraint delete'.  If
        --wait is specified, pcs will wait up to 'n' seconds for the resource
        to move and then return 0 on success or 1 on error.  If 'n' is not
        specified it defaults to 60 minutes.
        If you want the resource to preferably avoid running on some nodes but
        be able to failover to them use 'pcs constraint location avoids'.

    ban <resource id> [node] [--master] [lifetime=<lifetime>] [--wait[=n]]
        Prevent the resource id specified from running on the node (or on the
        current node it is running on if no node is specified) by creating a
        -INFINITY location constraint.  If --master is used the scope of the
        command is limited to the master role and you must use the master id
        (instead of the resource id).  If lifetime is specified then the
        constraint will expire after that time, otherwise it defaults to
        infinity and the constraint can be cleared manually with 'pcs resource
        clear' or 'pcs constraint delete'.  If --wait is specified, pcs will
        wait up to 'n' seconds for the resource to move and then return 0
        on success or 1 on error. If 'n' is not specified it defaults to 60
        minutes.
        If you want the resource to preferably avoid running on some nodes but
        be able to failover to them use 'pcs constraint location avoids'.

    clear <resource id> [node] [--master] [--wait[=n]]
        Remove constraints created by move and/or ban on the specified
        resource (and node if specified).
        If --master is used the scope of the command is limited to the
        master role and you must use the master id (instead of the resource id).
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including starting and/or moving resources if
        appropriate) and then return 0 on success or 1 on error.  If 'n' is not
        specified it defaults to 60 minutes.

    standards
        List available resource agent standards supported by this installation
        (OCF, LSB, etc.).

    providers
        List available OCF resource agent providers.

    agents [standard[:provider]]
        List available agents optionally filtered by standard and provider.

    update <resource id> [resource options] [op [<operation action>
           <operation options>]...] [meta <meta operations>...] [--wait[=n]]
        Add/Change options to specified resource, clone or multi-state
        resource.  If an operation (op) is specified it will update the first
        found operation with the same action on the specified resource, if no
        operation with that action exists then a new operation will be created.
        (WARNING: all existing options on the updated operation will be reset
        if not specified.)  If you want to create multiple monitor operations
        you should use the 'op add' & 'op remove' commands.  If --wait is
        specified, pcs will wait up to 'n' seconds for the changes to take
        effect and then return 0 if the changes have been processed or 1
        otherwise.  If 'n' is not specified it defaults to 60 minutes.

    op add <resource id> <operation action> [operation properties]
        Add operation for specified resource.

    op remove <resource id> <operation action> [<operation properties>...]
        Remove specified operation (note: you must specify the exact operation
        properties to properly remove an existing operation).

    op remove <operation id>
        Remove the specified operation id.

    op defaults [options]
        Set default values for operations, if no options are passed, lists
        currently configured defaults. Defaults do not apply to resources which
        override them with their own defined operations.

    meta <resource id | group id | master id | clone id> <meta options>
         [--wait[=n]]
        Add specified options to the specified resource, group, master/slave
        or clone.  Meta options should be in the format of name=value, options
        may be removed by setting an option without a value.  If --wait is
        specified, pcs will wait up to 'n' seconds for the changes to take
        effect and then return 0 if the changes have been processed or 1
        otherwise.  If 'n' is not specified it defaults to 60 minutes.
        Example: pcs resource meta TestResource failure-timeout=50 stickiness=

    group add <group id> <resource id> [resource id] ... [resource id]
              [--before <resource id> | --after <resource id>] [--wait[=n]]
        Add the specified resource to the group, creating the group if it does
        not exist. If the resource is present in another group it is moved to
        the new group. You can use --before or --after to specify the position
        of the added resources relatively to some resource already existing in
        the group. By adding resources to a group they are already in and
        specifying --after or --before you can move the resources in the group.
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including moving resources if appropriate) and
        then return 0 on success or 1 on error. If 'n' is not specified it
        defaults to 60 minutes.

    group remove <group id> <resource id> [resource id] ... [resource id]
          [--wait[=n]]
        Remove the specified resource(s) from the group, removing the group if
        no resources remain in it. If --wait is specified, pcs will wait up to
        'n' seconds for the operation to finish (including moving resources if
        appropriate) and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    ungroup <group id> [resource id] ... [resource id] [--wait[=n]]
        Remove the group (note: this does not remove any resources from the
        cluster) or if resources are specified, remove the specified resources
        from the group.  If --wait is specified, pcs will wait up to 'n' seconds
        for the operation to finish (including moving resources if appropriate)
        and the return 0 on success or 1 on error.  If 'n' is not specified it
        defaults to 60 minutes.

    clone <resource id | group id> [clone options]... [--wait[=n]]
        Set up the specified resource or group as a clone. If --wait is
        specified, pcs will wait up to 'n' seconds for the operation to finish
        (including starting clone instances if appropriate) and then return 0
        on success or 1 on error. If 'n' is not specified it defaults to 60
        minutes.

    unclone <resource id | group id> [--wait[=n]]
        Remove the clone which contains the specified group or resource (the
        resource or group will not be removed).  If --wait is specified, pcs
        will wait up to 'n' seconds for the operation to finish (including
        stopping clone instances if appropriate) and then return 0 on success
        or 1 on error.  If 'n' is not specified it defaults to 60 minutes.

    master [<master/slave id>] <resource id | group id> [options] [--wait[=n]]
        Configure a resource or group as a multi-state (master/slave) resource.
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including starting and promoting resource
        instances if appropriate) and then return 0 on success or 1 on error.
        If 'n' is not specified it defaults to 60 minutes.
        Note: to remove a master you must remove the resource/group it contains.

    bundle create <bundle id> container <container type> [<container options>]
            [network <network options>] [port-map <port options>]...
            [storage-map <storage options>]... [meta <meta options>]
            [--disabled] [--wait[=n]]
        Create a new bundle encapsulating no resources. The bundle can be used
        either as it is or a resource may be put into it at any time.
        If --disabled is specified, the bundle is not started automatically.
        If --wait is specified, pcs will wait up to 'n' seconds for the bundle
        to start and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    bundle reset <bundle id> [container <container options>]
            [network <network options>] [port-map <port options>]...
            [storage-map <storage options>]... [meta <meta options>]
            [--disabled] [--wait[=n]]
        Configure specified bundle with given options. Unlike bundle update,
        this command resets the bundle according given options - no previous
        options are kept. Resources inside the bundle are kept as they are.
        If --disabled is specified, the bundle is not started automatically.
        If --wait is specified, pcs will wait up to 'n' seconds for the bundle
        to start and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    bundle update <bundle id> [container <container options>]
            [network <network options>]
            [port-map (add <port options>) | (remove <id>...)]...
            [storage-map (add <storage options>) | (remove <id>...)]...
            [meta <meta options>]
            [--wait[=n]]
        Add, remove or change options to specified bundle. If you wish to update
        a resource encapsulated in the bundle, use the 'pcs resource update'
        command instead and specify the resource id. If --wait is specified,
        pcs will wait up to 'n' seconds for the operation to finish (including
        moving resources if appropriate) and then return 0 on success or 1 on
        error. If 'n' is not specified it defaults to 60 minutes.

    manage <resource id>... [--monitor]
        Set resources listed to managed mode (default). If --monitor is
        specified, enable all monitor operations of the resources.

    unmanage <resource id>... [--monitor]
        Set resources listed to unmanaged mode. When a resource is in unmanaged
        mode, the cluster is not allowed to start nor stop the resource. If
        --monitor is specified, disable all monitor operations of the
        resources.

    defaults [options]
        Set default values for resources, if no options are passed, lists
        currently configured defaults. Defaults do not apply to resources which
        override them with their own defined values.

    cleanup [<resource id>] [--node <node>]
        Make the cluster forget failed operations from history of the resource
        and re-detect its current state. This can be useful to purge knowledge
        of past failures that have since been resolved. If a resource id is not
        specified then all resources / stonith devices will be cleaned up. If a
        node is not specified then resources / stonith devices on all nodes will
        be cleaned up.

    refresh [<resource id>] [--node <node>] [--full]
        Make the cluster forget the complete operation history (including
        failures) of the resource and re-detect its current state. If you are
        interested in forgetting failed operations only, use the 'pcs resource
        cleanup' command. If a resource id is not specified then all resources
        / stonith devices will be refreshed. If a node is not specified then
        resources / stonith devices on all nodes will be refreshed. Use --full
        to refresh a resource on all nodes, otherwise only nodes where the
        resource's state is known will be considered.

    failcount show [<resource id> [<node> [<operation> [<interval>]]]] [--full]
        Show current failcount for resources, optionally filtered by a resource,
        node, operation and its interval. If --full is specified do not sum
        failcounts per resource and node. Operation, interval and --full
        options require pacemaker-1.1.18 or newer.

    failcount reset [<resource id> [<node> [<operation> [interval=<interval>]]]]
        Reset failcount for specified resource on all nodes or only on specified
        node. This tells the cluster to forget how many times a resource has
        failed in the past. This may allow the resource to be started or moved
        to a more preferred location. Operation and interval options require
        pacemaker-1.1.18 or newer.

    relocate dry-run [resource1] [resource2] ...
        The same as 'relocate run' but has no effect on the cluster.

    relocate run [resource1] [resource2] ...
        Relocate specified resources to their preferred nodes.  If no resources
        are specified, relocate all resources.
        This command calculates the preferred node for each resource while
        ignoring resource stickiness.  Then it creates location constraints
        which will cause the resources to move to their preferred nodes.  Once
        the resources have been moved the constraints are deleted automatically.
        Note that the preferred node is calculated based on current cluster
        status, constraints, location of resources and other settings and thus
        it might change over time.

    relocate show
        Display current status of resources and their optimal node ignoring
        resource stickiness.

    relocate clear
        Remove all constraints created by the 'relocate run' command.

    utilization [<resource id> [<name>=<value> ...]]
        Add specified utilization options to specified resource. If resource is
        not specified, shows utilization of all resources. If utilization
        options are not specified, shows utilization of specified resource.
        Utilization option should be in format name=value, value has to be
        integer. Options may be removed by setting an option without a value.
        Example: pcs resource utilization TestResource cpu= ram=20

Examples:

    pcs resource show
      Show all resources.

    pcs resource show VirtualIP
      Show options specific to the 'VirtualIP' resource.


    pcs resource create VirtualIP ocf:heartbeat:IPaddr2 ip=192.168.0.99 \
               cidr_netmask=32 nic=eth2 op monitor interval=30s
      Create a new resource called 'VirtualIP' with options.

    pcs resource create VirtualIP IPaddr2 ip=192.168.0.99 \
               cidr_netmask=32 nic=eth2 op monitor interval=30s
      Create a new resource called 'VirtualIP' with options.

    pcs resource update VirtualIP ip=192.168.0.98 nic=
      Change the ip address of VirtualIP and remove the nic option.

    pcs resource delete VirtualIP
      Delete the VirtualIP resource.

Notes:
    Starting resources on a cluster is (almost) always done by pacemaker and
    not directly from pcs.  If your resource isn't starting, it's usually
    due to either a misconfiguration of the resource (which you debug in
    the system log), or constraints preventing the resource from starting or
    the resource being disabled.  You can use 'pcs resource debug-start' to
    test resource configuration, but it should *not* normally be used to start
    resources in a cluster.


[root@cluster1 ~]# pcs resource run halvm

Usage: pcs resource [commands]...
Manage pacemaker resources

Commands:
    [show [<resource id>] | --full | --groups | --hide-inactive]
        Show all currently configured resources or if a resource is specified
        show the options for the configured resource.  If --full is specified,
        all configured resource options will be displayed.  If --groups is
        specified, only show groups (and their resources).  If --hide-inactive
        is specified, only show active resources.

    list [filter] [--nodesc]
        Show list of all available resource agents (if filter is provided then
        only resource agents matching the filter will be shown). If --nodesc is
        used then descriptions of resource agents are not printed.

    describe [<standard>:[<provider>:]]<type> [--full]
        Show options for the specified resource. If --full is specified, all
        options including advanced ones are shown.

    create <resource id> [<standard>:[<provider>:]]<type> [resource options]
           [op <operation action> <operation options> [<operation action>
           <operation options>]...] [meta <meta options>...]
           [clone [<clone options>] | master [<master options>] |
           --group <group id> [--before <resource id> | --after <resource id>]
           | bundle <bundle id>] [--disabled] [--no-default-ops] [--wait[=n]]
        Create specified resource. If clone is used a clone resource is
        created. If master is specified a master/slave resource is created.
        If --group is specified the resource is added to the group named. You
        can use --before or --after to specify the position of the added
        resource relatively to some resource already existing in the group.
        If bundle is used, the resource will be created inside of the specified
        bundle. If --disabled is specified the resource is not started
        automatically. If --no-default-ops is specified, only monitor
        operations are created for the resource and all other operations use
        default settings. If --wait is specified, pcs will wait up to 'n'
        seconds for the resource to start and then return 0 if the resource is
        started, or 1 if the resource has not yet started. If 'n' is not
        specified it defaults to 60 minutes.
        Example: Create a new resource called 'VirtualIP' with IP address
            192.168.0.99, netmask of 32, monitored everything 30 seconds,
            on eth2:
            pcs resource create VirtualIP ocf:heartbeat:IPaddr2 \
                ip=192.168.0.99 cidr_netmask=32 nic=eth2 \
                op monitor interval=30s

    delete <resource id|group id|master id|clone id>
        Deletes the resource, group, master or clone (and all resources within
        the group/master/clone).

    enable <resource id>... [--wait[=n]]
        Allow the cluster to start the resources. Depending on the rest of the
        configuration (constraints, options, failures, etc), the resources may
        remain stopped. If --wait is specified, pcs will wait up to 'n' seconds
        for the resources to start and then return 0 if the resources are
        started, or 1 if the resources have not yet started. If 'n' is not
        specified it defaults to 60 minutes.

    disable <resource id>... [--wait[=n]]
        Attempt to stop the resources if they are running and forbid the
        cluster from starting them again. Depending on the rest of the
        configuration (constraints, options, failures, etc), the resources may
        remain started. If --wait is specified, pcs will wait up to 'n' seconds
        for the resources to stop and then return 0 if the resources are
        stopped or 1 if the resources have not stopped. If 'n' is not specified
        it defaults to 60 minutes.

    restart <resource id> [node] [--wait=n]
        Restart the resource specified. If a node is specified and if the
        resource is a clone or master/slave it will be restarted only on
        the node specified.  If --wait is specified, then we will wait
        up to 'n' seconds for the resource to be restarted and return 0 if
        the restart was successful or 1 if it was not.

    debug-start <resource id> [--full]
        This command will force the specified resource to start on this node
        ignoring the cluster recommendations and print the output from
        starting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to start.

    debug-stop <resource id> [--full]
        This command will force the specified resource to stop on this node
        ignoring the cluster recommendations and print the output from
        stopping the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to stop.

    debug-promote <resource id> [--full]
        This command will force the specified resource to be promoted on this
        node ignoring the cluster recommendations and print the output from
        promoting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to promote.

    debug-demote <resource id> [--full]
        This command will force the specified resource to be demoted on this
        node ignoring the cluster recommendations and print the output from
        demoting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to demote.

    debug-monitor <resource id> [--full]
        This command will force the specified resource to be monitored on this
        node  ignoring the cluster recommendations and print the output from
        monitoring the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to be monitored.

    move <resource id> [destination node] [--master] [lifetime=<lifetime>]
         [--wait[=n]]
        Move the resource off the node it is currently running on by creating a
        -INFINITY location constraint to ban the node.  If destination node is
        specified the resource will be moved to that node by creating an
        INFINITY location constraint to prefer the destination node.  If
        --master is used the scope of the command is limited to the master role
        and you must use the master id (instead of the resource id).  If
        lifetime is specified then the constraint will expire after that time,
        otherwise it defaults to infinity and the constraint can be cleared
        manually with 'pcs resource clear' or 'pcs constraint delete'.  If
        --wait is specified, pcs will wait up to 'n' seconds for the resource
        to move and then return 0 on success or 1 on error.  If 'n' is not
        specified it defaults to 60 minutes.
        If you want the resource to preferably avoid running on some nodes but
        be able to failover to them use 'pcs constraint location avoids'.

    ban <resource id> [node] [--master] [lifetime=<lifetime>] [--wait[=n]]
        Prevent the resource id specified from running on the node (or on the
        current node it is running on if no node is specified) by creating a
        -INFINITY location constraint.  If --master is used the scope of the
        command is limited to the master role and you must use the master id
        (instead of the resource id).  If lifetime is specified then the
        constraint will expire after that time, otherwise it defaults to
        infinity and the constraint can be cleared manually with 'pcs resource
        clear' or 'pcs constraint delete'.  If --wait is specified, pcs will
        wait up to 'n' seconds for the resource to move and then return 0
        on success or 1 on error. If 'n' is not specified it defaults to 60
        minutes.
        If you want the resource to preferably avoid running on some nodes but
        be able to failover to them use 'pcs constraint location avoids'.

    clear <resource id> [node] [--master] [--wait[=n]]
        Remove constraints created by move and/or ban on the specified
        resource (and node if specified).
        If --master is used the scope of the command is limited to the
        master role and you must use the master id (instead of the resource id).
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including starting and/or moving resources if
        appropriate) and then return 0 on success or 1 on error.  If 'n' is not
        specified it defaults to 60 minutes.

    standards
        List available resource agent standards supported by this installation
        (OCF, LSB, etc.).

    providers
        List available OCF resource agent providers.

    agents [standard[:provider]]
        List available agents optionally filtered by standard and provider.

    update <resource id> [resource options] [op [<operation action>
           <operation options>]...] [meta <meta operations>...] [--wait[=n]]
        Add/Change options to specified resource, clone or multi-state
        resource.  If an operation (op) is specified it will update the first
        found operation with the same action on the specified resource, if no
        operation with that action exists then a new operation will be created.
        (WARNING: all existing options on the updated operation will be reset
        if not specified.)  If you want to create multiple monitor operations
        you should use the 'op add' & 'op remove' commands.  If --wait is
        specified, pcs will wait up to 'n' seconds for the changes to take
        effect and then return 0 if the changes have been processed or 1
        otherwise.  If 'n' is not specified it defaults to 60 minutes.

    op add <resource id> <operation action> [operation properties]
        Add operation for specified resource.

    op remove <resource id> <operation action> [<operation properties>...]
        Remove specified operation (note: you must specify the exact operation
        properties to properly remove an existing operation).

    op remove <operation id>
        Remove the specified operation id.

    op defaults [options]
        Set default values for operations, if no options are passed, lists
        currently configured defaults. Defaults do not apply to resources which
        override them with their own defined operations.

    meta <resource id | group id | master id | clone id> <meta options>
         [--wait[=n]]
        Add specified options to the specified resource, group, master/slave
        or clone.  Meta options should be in the format of name=value, options
        may be removed by setting an option without a value.  If --wait is
        specified, pcs will wait up to 'n' seconds for the changes to take
        effect and then return 0 if the changes have been processed or 1
        otherwise.  If 'n' is not specified it defaults to 60 minutes.
        Example: pcs resource meta TestResource failure-timeout=50 stickiness=

    group add <group id> <resource id> [resource id] ... [resource id]
              [--before <resource id> | --after <resource id>] [--wait[=n]]
        Add the specified resource to the group, creating the group if it does
        not exist. If the resource is present in another group it is moved to
        the new group. You can use --before or --after to specify the position
        of the added resources relatively to some resource already existing in
        the group. By adding resources to a group they are already in and
        specifying --after or --before you can move the resources in the group.
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including moving resources if appropriate) and
        then return 0 on success or 1 on error. If 'n' is not specified it
        defaults to 60 minutes.

    group remove <group id> <resource id> [resource id] ... [resource id]
          [--wait[=n]]
        Remove the specified resource(s) from the group, removing the group if
        no resources remain in it. If --wait is specified, pcs will wait up to
        'n' seconds for the operation to finish (including moving resources if
        appropriate) and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    ungroup <group id> [resource id] ... [resource id] [--wait[=n]]
        Remove the group (note: this does not remove any resources from the
        cluster) or if resources are specified, remove the specified resources
        from the group.  If --wait is specified, pcs will wait up to 'n' seconds
        for the operation to finish (including moving resources if appropriate)
        and the return 0 on success or 1 on error.  If 'n' is not specified it
        defaults to 60 minutes.

    clone <resource id | group id> [clone options]... [--wait[=n]]
        Set up the specified resource or group as a clone. If --wait is
        specified, pcs will wait up to 'n' seconds for the operation to finish
        (including starting clone instances if appropriate) and then return 0
        on success or 1 on error. If 'n' is not specified it defaults to 60
        minutes.

    unclone <resource id | group id> [--wait[=n]]
        Remove the clone which contains the specified group or resource (the
        resource or group will not be removed).  If --wait is specified, pcs
        will wait up to 'n' seconds for the operation to finish (including
        stopping clone instances if appropriate) and then return 0 on success
        or 1 on error.  If 'n' is not specified it defaults to 60 minutes.

    master [<master/slave id>] <resource id | group id> [options] [--wait[=n]]
        Configure a resource or group as a multi-state (master/slave) resource.
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including starting and promoting resource
        instances if appropriate) and then return 0 on success or 1 on error.
        If 'n' is not specified it defaults to 60 minutes.
        Note: to remove a master you must remove the resource/group it contains.

    bundle create <bundle id> container <container type> [<container options>]
            [network <network options>] [port-map <port options>]...
            [storage-map <storage options>]... [meta <meta options>]
            [--disabled] [--wait[=n]]
        Create a new bundle encapsulating no resources. The bundle can be used
        either as it is or a resource may be put into it at any time.
        If --disabled is specified, the bundle is not started automatically.
        If --wait is specified, pcs will wait up to 'n' seconds for the bundle
        to start and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    bundle reset <bundle id> [container <container options>]
            [network <network options>] [port-map <port options>]...
            [storage-map <storage options>]... [meta <meta options>]
            [--disabled] [--wait[=n]]
        Configure specified bundle with given options. Unlike bundle update,
        this command resets the bundle according given options - no previous
        options are kept. Resources inside the bundle are kept as they are.
        If --disabled is specified, the bundle is not started automatically.
        If --wait is specified, pcs will wait up to 'n' seconds for the bundle
        to start and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    bundle update <bundle id> [container <container options>]
            [network <network options>]
            [port-map (add <port options>) | (remove <id>...)]...
            [storage-map (add <storage options>) | (remove <id>...)]...
            [meta <meta options>]
            [--wait[=n]]
        Add, remove or change options to specified bundle. If you wish to update
        a resource encapsulated in the bundle, use the 'pcs resource update'
        command instead and specify the resource id. If --wait is specified,
        pcs will wait up to 'n' seconds for the operation to finish (including
        moving resources if appropriate) and then return 0 on success or 1 on
        error. If 'n' is not specified it defaults to 60 minutes.

    manage <resource id>... [--monitor]
        Set resources listed to managed mode (default). If --monitor is
        specified, enable all monitor operations of the resources.

    unmanage <resource id>... [--monitor]
        Set resources listed to unmanaged mode. When a resource is in unmanaged
        mode, the cluster is not allowed to start nor stop the resource. If
        --monitor is specified, disable all monitor operations of the
        resources.

    defaults [options]
        Set default values for resources, if no options are passed, lists
        currently configured defaults. Defaults do not apply to resources which
        override them with their own defined values.

    cleanup [<resource id>] [--node <node>]
        Make the cluster forget failed operations from history of the resource
        and re-detect its current state. This can be useful to purge knowledge
        of past failures that have since been resolved. If a resource id is not
        specified then all resources / stonith devices will be cleaned up. If a
        node is not specified then resources / stonith devices on all nodes will
        be cleaned up.

    refresh [<resource id>] [--node <node>] [--full]
        Make the cluster forget the complete operation history (including
        failures) of the resource and re-detect its current state. If you are
        interested in forgetting failed operations only, use the 'pcs resource
        cleanup' command. If a resource id is not specified then all resources
        / stonith devices will be refreshed. If a node is not specified then
        resources / stonith devices on all nodes will be refreshed. Use --full
        to refresh a resource on all nodes, otherwise only nodes where the
        resource's state is known will be considered.

    failcount show [<resource id> [<node> [<operation> [<interval>]]]] [--full]
        Show current failcount for resources, optionally filtered by a resource,
        node, operation and its interval. If --full is specified do not sum
        failcounts per resource and node. Operation, interval and --full
        options require pacemaker-1.1.18 or newer.

    failcount reset [<resource id> [<node> [<operation> [interval=<interval>]]]]
        Reset failcount for specified resource on all nodes or only on specified
        node. This tells the cluster to forget how many times a resource has
        failed in the past. This may allow the resource to be started or moved
        to a more preferred location. Operation and interval options require
        pacemaker-1.1.18 or newer.

    relocate dry-run [resource1] [resource2] ...
        The same as 'relocate run' but has no effect on the cluster.

    relocate run [resource1] [resource2] ...
        Relocate specified resources to their preferred nodes.  If no resources
        are specified, relocate all resources.
        This command calculates the preferred node for each resource while
        ignoring resource stickiness.  Then it creates location constraints
        which will cause the resources to move to their preferred nodes.  Once
        the resources have been moved the constraints are deleted automatically.
        Note that the preferred node is calculated based on current cluster
        status, constraints, location of resources and other settings and thus
        it might change over time.

    relocate show
        Display current status of resources and their optimal node ignoring
        resource stickiness.

    relocate clear
        Remove all constraints created by the 'relocate run' command.

    utilization [<resource id> [<name>=<value> ...]]
        Add specified utilization options to specified resource. If resource is
        not specified, shows utilization of all resources. If utilization
        options are not specified, shows utilization of specified resource.
        Utilization option should be in format name=value, value has to be
        integer. Options may be removed by setting an option without a value.
        Example: pcs resource utilization TestResource cpu= ram=20

Examples:

    pcs resource show
      Show all resources.

    pcs resource show VirtualIP
      Show options specific to the 'VirtualIP' resource.


    pcs resource create VirtualIP ocf:heartbeat:IPaddr2 ip=192.168.0.99 \
               cidr_netmask=32 nic=eth2 op monitor interval=30s
      Create a new resource called 'VirtualIP' with options.

    pcs resource create VirtualIP IPaddr2 ip=192.168.0.99 \
               cidr_netmask=32 nic=eth2 op monitor interval=30s
      Create a new resource called 'VirtualIP' with options.

    pcs resource update VirtualIP ip=192.168.0.98 nic=
      Change the ip address of VirtualIP and remove the nic option.

    pcs resource delete VirtualIP
      Delete the VirtualIP resource.

Notes:
    Starting resources on a cluster is (almost) always done by pacemaker and
    not directly from pcs.  If your resource isn't starting, it's usually
    due to either a misconfiguration of the resource (which you debug in
    the system log), or constraints preventing the resource from starting or
    the resource being disabled.  You can use 'pcs resource debug-start' to
    test resource configuration, but it should *not* normally be used to start
    resources in a cluster.


[root@cluster1 ~]# pcs resource halvm

Usage: pcs resource [commands]...
Manage pacemaker resources

Commands:
    [show [<resource id>] | --full | --groups | --hide-inactive]
        Show all currently configured resources or if a resource is specified
        show the options for the configured resource.  If --full is specified,
        all configured resource options will be displayed.  If --groups is
        specified, only show groups (and their resources).  If --hide-inactive
        is specified, only show active resources.

    list [filter] [--nodesc]
        Show list of all available resource agents (if filter is provided then
        only resource agents matching the filter will be shown). If --nodesc is
        used then descriptions of resource agents are not printed.

    describe [<standard>:[<provider>:]]<type> [--full]
        Show options for the specified resource. If --full is specified, all
        options including advanced ones are shown.

    create <resource id> [<standard>:[<provider>:]]<type> [resource options]
           [op <operation action> <operation options> [<operation action>
           <operation options>]...] [meta <meta options>...]
           [clone [<clone options>] | master [<master options>] |
           --group <group id> [--before <resource id> | --after <resource id>]
           | bundle <bundle id>] [--disabled] [--no-default-ops] [--wait[=n]]
        Create specified resource. If clone is used a clone resource is
        created. If master is specified a master/slave resource is created.
        If --group is specified the resource is added to the group named. You
        can use --before or --after to specify the position of the added
        resource relatively to some resource already existing in the group.
        If bundle is used, the resource will be created inside of the specified
        bundle. If --disabled is specified the resource is not started
        automatically. If --no-default-ops is specified, only monitor
        operations are created for the resource and all other operations use
        default settings. If --wait is specified, pcs will wait up to 'n'
        seconds for the resource to start and then return 0 if the resource is
        started, or 1 if the resource has not yet started. If 'n' is not
        specified it defaults to 60 minutes.
        Example: Create a new resource called 'VirtualIP' with IP address
            192.168.0.99, netmask of 32, monitored everything 30 seconds,
            on eth2:
            pcs resource create VirtualIP ocf:heartbeat:IPaddr2 \
                ip=192.168.0.99 cidr_netmask=32 nic=eth2 \
                op monitor interval=30s

    delete <resource id|group id|master id|clone id>
        Deletes the resource, group, master or clone (and all resources within
        the group/master/clone).

    enable <resource id>... [--wait[=n]]
        Allow the cluster to start the resources. Depending on the rest of the
        configuration (constraints, options, failures, etc), the resources may
        remain stopped. If --wait is specified, pcs will wait up to 'n' seconds
        for the resources to start and then return 0 if the resources are
        started, or 1 if the resources have not yet started. If 'n' is not
        specified it defaults to 60 minutes.

    disable <resource id>... [--wait[=n]]
        Attempt to stop the resources if they are running and forbid the
        cluster from starting them again. Depending on the rest of the
        configuration (constraints, options, failures, etc), the resources may
        remain started. If --wait is specified, pcs will wait up to 'n' seconds
        for the resources to stop and then return 0 if the resources are
        stopped or 1 if the resources have not stopped. If 'n' is not specified
        it defaults to 60 minutes.

    restart <resource id> [node] [--wait=n]
        Restart the resource specified. If a node is specified and if the
        resource is a clone or master/slave it will be restarted only on
        the node specified.  If --wait is specified, then we will wait
        up to 'n' seconds for the resource to be restarted and return 0 if
        the restart was successful or 1 if it was not.

    debug-start <resource id> [--full]
        This command will force the specified resource to start on this node
        ignoring the cluster recommendations and print the output from
        starting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to start.

    debug-stop <resource id> [--full]
        This command will force the specified resource to stop on this node
        ignoring the cluster recommendations and print the output from
        stopping the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to stop.

    debug-promote <resource id> [--full]
        This command will force the specified resource to be promoted on this
        node ignoring the cluster recommendations and print the output from
        promoting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to promote.

    debug-demote <resource id> [--full]
        This command will force the specified resource to be demoted on this
        node ignoring the cluster recommendations and print the output from
        demoting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to demote.

    debug-monitor <resource id> [--full]
        This command will force the specified resource to be monitored on this
        node  ignoring the cluster recommendations and print the output from
        monitoring the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to be monitored.

    move <resource id> [destination node] [--master] [lifetime=<lifetime>]
         [--wait[=n]]
        Move the resource off the node it is currently running on by creating a
        -INFINITY location constraint to ban the node.  If destination node is
        specified the resource will be moved to that node by creating an
        INFINITY location constraint to prefer the destination node.  If
        --master is used the scope of the command is limited to the master role
        and you must use the master id (instead of the resource id).  If
        lifetime is specified then the constraint will expire after that time,
        otherwise it defaults to infinity and the constraint can be cleared
        manually with 'pcs resource clear' or 'pcs constraint delete'.  If
        --wait is specified, pcs will wait up to 'n' seconds for the resource
        to move and then return 0 on success or 1 on error.  If 'n' is not
        specified it defaults to 60 minutes.
        If you want the resource to preferably avoid running on some nodes but
        be able to failover to them use 'pcs constraint location avoids'.

    ban <resource id> [node] [--master] [lifetime=<lifetime>] [--wait[=n]]
        Prevent the resource id specified from running on the node (or on the
        current node it is running on if no node is specified) by creating a
        -INFINITY location constraint.  If --master is used the scope of the
        command is limited to the master role and you must use the master id
        (instead of the resource id).  If lifetime is specified then the
        constraint will expire after that time, otherwise it defaults to
        infinity and the constraint can be cleared manually with 'pcs resource
        clear' or 'pcs constraint delete'.  If --wait is specified, pcs will
        wait up to 'n' seconds for the resource to move and then return 0
        on success or 1 on error. If 'n' is not specified it defaults to 60
        minutes.
        If you want the resource to preferably avoid running on some nodes but
        be able to failover to them use 'pcs constraint location avoids'.

    clear <resource id> [node] [--master] [--wait[=n]]
        Remove constraints created by move and/or ban on the specified
        resource (and node if specified).
        If --master is used the scope of the command is limited to the
        master role and you must use the master id (instead of the resource id).
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including starting and/or moving resources if
        appropriate) and then return 0 on success or 1 on error.  If 'n' is not
        specified it defaults to 60 minutes.

    standards
        List available resource agent standards supported by this installation
        (OCF, LSB, etc.).

    providers
        List available OCF resource agent providers.

    agents [standard[:provider]]
        List available agents optionally filtered by standard and provider.

    update <resource id> [resource options] [op [<operation action>
           <operation options>]...] [meta <meta operations>...] [--wait[=n]]
        Add/Change options to specified resource, clone or multi-state
        resource.  If an operation (op) is specified it will update the first
        found operation with the same action on the specified resource, if no
        operation with that action exists then a new operation will be created.
        (WARNING: all existing options on the updated operation will be reset
        if not specified.)  If you want to create multiple monitor operations
        you should use the 'op add' & 'op remove' commands.  If --wait is
        specified, pcs will wait up to 'n' seconds for the changes to take
        effect and then return 0 if the changes have been processed or 1
        otherwise.  If 'n' is not specified it defaults to 60 minutes.

    op add <resource id> <operation action> [operation properties]
        Add operation for specified resource.

    op remove <resource id> <operation action> [<operation properties>...]
        Remove specified operation (note: you must specify the exact operation
        properties to properly remove an existing operation).

    op remove <operation id>
        Remove the specified operation id.

    op defaults [options]
        Set default values for operations, if no options are passed, lists
        currently configured defaults. Defaults do not apply to resources which
        override them with their own defined operations.

    meta <resource id | group id | master id | clone id> <meta options>
         [--wait[=n]]
        Add specified options to the specified resource, group, master/slave
        or clone.  Meta options should be in the format of name=value, options
        may be removed by setting an option without a value.  If --wait is
        specified, pcs will wait up to 'n' seconds for the changes to take
        effect and then return 0 if the changes have been processed or 1
        otherwise.  If 'n' is not specified it defaults to 60 minutes.
        Example: pcs resource meta TestResource failure-timeout=50 stickiness=

    group add <group id> <resource id> [resource id] ... [resource id]
              [--before <resource id> | --after <resource id>] [--wait[=n]]
        Add the specified resource to the group, creating the group if it does
        not exist. If the resource is present in another group it is moved to
        the new group. You can use --before or --after to specify the position
        of the added resources relatively to some resource already existing in
        the group. By adding resources to a group they are already in and
        specifying --after or --before you can move the resources in the group.
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including moving resources if appropriate) and
        then return 0 on success or 1 on error. If 'n' is not specified it
        defaults to 60 minutes.

    group remove <group id> <resource id> [resource id] ... [resource id]
          [--wait[=n]]
        Remove the specified resource(s) from the group, removing the group if
        no resources remain in it. If --wait is specified, pcs will wait up to
        'n' seconds for the operation to finish (including moving resources if
        appropriate) and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    ungroup <group id> [resource id] ... [resource id] [--wait[=n]]
        Remove the group (note: this does not remove any resources from the
        cluster) or if resources are specified, remove the specified resources
        from the group.  If --wait is specified, pcs will wait up to 'n' seconds
        for the operation to finish (including moving resources if appropriate)
        and the return 0 on success or 1 on error.  If 'n' is not specified it
        defaults to 60 minutes.

    clone <resource id | group id> [clone options]... [--wait[=n]]
        Set up the specified resource or group as a clone. If --wait is
        specified, pcs will wait up to 'n' seconds for the operation to finish
        (including starting clone instances if appropriate) and then return 0
        on success or 1 on error. If 'n' is not specified it defaults to 60
        minutes.

    unclone <resource id | group id> [--wait[=n]]
        Remove the clone which contains the specified group or resource (the
        resource or group will not be removed).  If --wait is specified, pcs
        will wait up to 'n' seconds for the operation to finish (including
        stopping clone instances if appropriate) and then return 0 on success
        or 1 on error.  If 'n' is not specified it defaults to 60 minutes.

    master [<master/slave id>] <resource id | group id> [options] [--wait[=n]]
        Configure a resource or group as a multi-state (master/slave) resource.
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including starting and promoting resource
        instances if appropriate) and then return 0 on success or 1 on error.
        If 'n' is not specified it defaults to 60 minutes.
        Note: to remove a master you must remove the resource/group it contains.

    bundle create <bundle id> container <container type> [<container options>]
            [network <network options>] [port-map <port options>]...
            [storage-map <storage options>]... [meta <meta options>]
            [--disabled] [--wait[=n]]
        Create a new bundle encapsulating no resources. The bundle can be used
        either as it is or a resource may be put into it at any time.
        If --disabled is specified, the bundle is not started automatically.
        If --wait is specified, pcs will wait up to 'n' seconds for the bundle
        to start and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    bundle reset <bundle id> [container <container options>]
            [network <network options>] [port-map <port options>]...
            [storage-map <storage options>]... [meta <meta options>]
            [--disabled] [--wait[=n]]
        Configure specified bundle with given options. Unlike bundle update,
        this command resets the bundle according given options - no previous
        options are kept. Resources inside the bundle are kept as they are.
        If --disabled is specified, the bundle is not started automatically.
        If --wait is specified, pcs will wait up to 'n' seconds for the bundle
        to start and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    bundle update <bundle id> [container <container options>]
            [network <network options>]
            [port-map (add <port options>) | (remove <id>...)]...
            [storage-map (add <storage options>) | (remove <id>...)]...
            [meta <meta options>]
            [--wait[=n]]
        Add, remove or change options to specified bundle. If you wish to update
        a resource encapsulated in the bundle, use the 'pcs resource update'
        command instead and specify the resource id. If --wait is specified,
        pcs will wait up to 'n' seconds for the operation to finish (including
        moving resources if appropriate) and then return 0 on success or 1 on
        error. If 'n' is not specified it defaults to 60 minutes.

    manage <resource id>... [--monitor]
        Set resources listed to managed mode (default). If --monitor is
        specified, enable all monitor operations of the resources.

    unmanage <resource id>... [--monitor]
        Set resources listed to unmanaged mode. When a resource is in unmanaged
        mode, the cluster is not allowed to start nor stop the resource. If
        --monitor is specified, disable all monitor operations of the
        resources.

    defaults [options]
        Set default values for resources, if no options are passed, lists
        currently configured defaults. Defaults do not apply to resources which
        override them with their own defined values.

    cleanup [<resource id>] [--node <node>]
        Make the cluster forget failed operations from history of the resource
        and re-detect its current state. This can be useful to purge knowledge
        of past failures that have since been resolved. If a resource id is not
        specified then all resources / stonith devices will be cleaned up. If a
        node is not specified then resources / stonith devices on all nodes will
        be cleaned up.

    refresh [<resource id>] [--node <node>] [--full]
        Make the cluster forget the complete operation history (including
        failures) of the resource and re-detect its current state. If you are
        interested in forgetting failed operations only, use the 'pcs resource
        cleanup' command. If a resource id is not specified then all resources
        / stonith devices will be refreshed. If a node is not specified then
        resources / stonith devices on all nodes will be refreshed. Use --full
        to refresh a resource on all nodes, otherwise only nodes where the
        resource's state is known will be considered.

    failcount show [<resource id> [<node> [<operation> [<interval>]]]] [--full]
        Show current failcount for resources, optionally filtered by a resource,
        node, operation and its interval. If --full is specified do not sum
        failcounts per resource and node. Operation, interval and --full
        options require pacemaker-1.1.18 or newer.

    failcount reset [<resource id> [<node> [<operation> [interval=<interval>]]]]
        Reset failcount for specified resource on all nodes or only on specified
        node. This tells the cluster to forget how many times a resource has
        failed in the past. This may allow the resource to be started or moved
        to a more preferred location. Operation and interval options require
        pacemaker-1.1.18 or newer.

    relocate dry-run [resource1] [resource2] ...
        The same as 'relocate run' but has no effect on the cluster.

    relocate run [resource1] [resource2] ...
        Relocate specified resources to their preferred nodes.  If no resources
        are specified, relocate all resources.
        This command calculates the preferred node for each resource while
        ignoring resource stickiness.  Then it creates location constraints
        which will cause the resources to move to their preferred nodes.  Once
        the resources have been moved the constraints are deleted automatically.
        Note that the preferred node is calculated based on current cluster
        status, constraints, location of resources and other settings and thus
        it might change over time.

    relocate show
        Display current status of resources and their optimal node ignoring
        resource stickiness.

    relocate clear
        Remove all constraints created by the 'relocate run' command.

    utilization [<resource id> [<name>=<value> ...]]
        Add specified utilization options to specified resource. If resource is
        not specified, shows utilization of all resources. If utilization
        options are not specified, shows utilization of specified resource.
        Utilization option should be in format name=value, value has to be
        integer. Options may be removed by setting an option without a value.
        Example: pcs resource utilization TestResource cpu= ram=20

Examples:

    pcs resource show
      Show all resources.

    pcs resource show VirtualIP
      Show options specific to the 'VirtualIP' resource.


    pcs resource create VirtualIP ocf:heartbeat:IPaddr2 ip=192.168.0.99 \
               cidr_netmask=32 nic=eth2 op monitor interval=30s
      Create a new resource called 'VirtualIP' with options.

    pcs resource create VirtualIP IPaddr2 ip=192.168.0.99 \
               cidr_netmask=32 nic=eth2 op monitor interval=30s
      Create a new resource called 'VirtualIP' with options.

    pcs resource update VirtualIP ip=192.168.0.98 nic=
      Change the ip address of VirtualIP and remove the nic option.

    pcs resource delete VirtualIP
      Delete the VirtualIP resource.

Notes:
    Starting resources on a cluster is (almost) always done by pacemaker and
    not directly from pcs.  If your resource isn't starting, it's usually
    due to either a misconfiguration of the resource (which you debug in
    the system log), or constraints preventing the resource from starting or
    the resource being disabled.  You can use 'pcs resource debug-start' to
    test resource configuration, but it should *not* normally be used to start
    resources in a cluster.


[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 22:57:40 2019
Last change: Sat Oct 26 22:54:19 2019 by root via cibadmin on cluster2

2 nodes configured
2 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Resource Group: halvmfs
     halvm (ocf::heartbeat:LVM): Stopped
 xfsfs (ocf::heartbeat:Filesystem): Stopped

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs cluster start --all
cluster2: Unable to connect to cluster2, try setting higher timeout in --request-timeout option (Failed connect to cluster2:2224; Connection refused)
cluster2: Not starting cluster - node is unreachable
cluster1: Starting Cluster (corosync)...
cluster1: Starting Cluster (pacemaker)...
Error: unable to start all nodes
[root@cluster1 ~]# pcs cluster auth cluster2
Username: hacluster
Password:
Error: Unable to communicate with cluster2
[root@cluster1 ~]# ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        ether 02:42:21:88:4b:74  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.20.10.2  netmask 255.255.255.240  broadcast 172.20.10.15
        inet6 fe80::22e:b097:7a87:fdb7  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:bb:11:58  txqueuelen 1000  (Ethernet)
        RX packets 10814  bytes 1300282 (1.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 15143  bytes 1911582 (1.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 7233  bytes 947830 (925.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 7233  bytes 947830 (925.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:1a:19:6d  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@cluster1 ~]# ping 172.20.10.2
PING 172.20.10.2 (172.20.10.2) 56(84) bytes of data.
64 bytes from 172.20.10.2: icmp_seq=1 ttl=64 time=0.059 ms
64 bytes from 172.20.10.2: icmp_seq=2 ttl=64 time=0.114 ms
^C
--- 172.20.10.2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.059/0.086/0.114/0.029 ms
[root@cluster1 ~]# ping 172.20.10.3
PING 172.20.10.3 (172.20.10.3) 56(84) bytes of data.
64 bytes from 172.20.10.3: icmp_seq=1 ttl=64 time=0.600 ms
64 bytes from 172.20.10.3: icmp_seq=2 ttl=64 time=0.466 ms
^C
--- 172.20.10.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 0.466/0.533/0.600/0.067 ms
[root@cluster1 ~]# pcs cluster auth cluster2
cluster2: Already authorized
[root@cluster1 ~]# pcs cluster start --all
cluster1: Starting Cluster (corosync)...
cluster2: Starting Cluster (corosync)...
cluster1: Starting Cluster (pacemaker)...
cluster2: Starting Cluster (pacemaker)...
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:00:58 2019
Last change: Sat Oct 26 22:54:19 2019 by root via cibadmin on cluster2

2 nodes configured
2 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Resource Group: halvmfs
     halvm (ocf::heartbeat:LVM): Stopped
 xfsfs (ocf::heartbeat:Filesystem): Stopped

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs resource enable halvm
[root@cluster1 ~]# pcs resource enable xfsfs
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:01:55 2019
Last change: Sat Oct 26 22:54:19 2019 by root via cibadmin on cluster2

2 nodes configured
2 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Resource Group: halvmfs
     halvm (ocf::heartbeat:LVM): Stopped
 xfsfs (ocf::heartbeat:Filesystem): Stopped

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs resource start halvm

Usage: pcs resource [commands]...
Manage pacemaker resources

Commands:
    [show [<resource id>] | --full | --groups | --hide-inactive]
        Show all currently configured resources or if a resource is specified
        show the options for the configured resource.  If --full is specified,
        all configured resource options will be displayed.  If --groups is
        specified, only show groups (and their resources).  If --hide-inactive
        is specified, only show active resources.

    list [filter] [--nodesc]
        Show list of all available resource agents (if filter is provided then
        only resource agents matching the filter will be shown). If --nodesc is
        used then descriptions of resource agents are not printed.

    describe [<standard>:[<provider>:]]<type> [--full]
        Show options for the specified resource. If --full is specified, all
        options including advanced ones are shown.

    create <resource id> [<standard>:[<provider>:]]<type> [resource options]
           [op <operation action> <operation options> [<operation action>
           <operation options>]...] [meta <meta options>...]
           [clone [<clone options>] | master [<master options>] |
           --group <group id> [--before <resource id> | --after <resource id>]
           | bundle <bundle id>] [--disabled] [--no-default-ops] [--wait[=n]]
        Create specified resource. If clone is used a clone resource is
        created. If master is specified a master/slave resource is created.
        If --group is specified the resource is added to the group named. You
        can use --before or --after to specify the position of the added
        resource relatively to some resource already existing in the group.
        If bundle is used, the resource will be created inside of the specified
        bundle. If --disabled is specified the resource is not started
        automatically. If --no-default-ops is specified, only monitor
        operations are created for the resource and all other operations use
        default settings. If --wait is specified, pcs will wait up to 'n'
        seconds for the resource to start and then return 0 if the resource is
        started, or 1 if the resource has not yet started. If 'n' is not
        specified it defaults to 60 minutes.
        Example: Create a new resource called 'VirtualIP' with IP address
            192.168.0.99, netmask of 32, monitored everything 30 seconds,
            on eth2:
            pcs resource create VirtualIP ocf:heartbeat:IPaddr2 \
                ip=192.168.0.99 cidr_netmask=32 nic=eth2 \
                op monitor interval=30s

    delete <resource id|group id|master id|clone id>
        Deletes the resource, group, master or clone (and all resources within
        the group/master/clone).

    enable <resource id>... [--wait[=n]]
        Allow the cluster to start the resources. Depending on the rest of the
        configuration (constraints, options, failures, etc), the resources may
        remain stopped. If --wait is specified, pcs will wait up to 'n' seconds
        for the resources to start and then return 0 if the resources are
        started, or 1 if the resources have not yet started. If 'n' is not
        specified it defaults to 60 minutes.

    disable <resource id>... [--wait[=n]]
        Attempt to stop the resources if they are running and forbid the
        cluster from starting them again. Depending on the rest of the
        configuration (constraints, options, failures, etc), the resources may
        remain started. If --wait is specified, pcs will wait up to 'n' seconds
        for the resources to stop and then return 0 if the resources are
        stopped or 1 if the resources have not stopped. If 'n' is not specified
        it defaults to 60 minutes.

    restart <resource id> [node] [--wait=n]
        Restart the resource specified. If a node is specified and if the
        resource is a clone or master/slave it will be restarted only on
        the node specified.  If --wait is specified, then we will wait
        up to 'n' seconds for the resource to be restarted and return 0 if
        the restart was successful or 1 if it was not.

    debug-start <resource id> [--full]
        This command will force the specified resource to start on this node
        ignoring the cluster recommendations and print the output from
        starting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to start.

    debug-stop <resource id> [--full]
        This command will force the specified resource to stop on this node
        ignoring the cluster recommendations and print the output from
        stopping the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to stop.

    debug-promote <resource id> [--full]
        This command will force the specified resource to be promoted on this
        node ignoring the cluster recommendations and print the output from
        promoting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to promote.

    debug-demote <resource id> [--full]
        This command will force the specified resource to be demoted on this
        node ignoring the cluster recommendations and print the output from
        demoting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to demote.

    debug-monitor <resource id> [--full]
        This command will force the specified resource to be monitored on this
        node  ignoring the cluster recommendations and print the output from
        monitoring the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to be monitored.

    move <resource id> [destination node] [--master] [lifetime=<lifetime>]
         [--wait[=n]]
        Move the resource off the node it is currently running on by creating a
        -INFINITY location constraint to ban the node.  If destination node is
        specified the resource will be moved to that node by creating an
        INFINITY location constraint to prefer the destination node.  If
        --master is used the scope of the command is limited to the master role
        and you must use the master id (instead of the resource id).  If
        lifetime is specified then the constraint will expire after that time,
        otherwise it defaults to infinity and the constraint can be cleared
        manually with 'pcs resource clear' or 'pcs constraint delete'.  If
        --wait is specified, pcs will wait up to 'n' seconds for the resource
        to move and then return 0 on success or 1 on error.  If 'n' is not
        specified it defaults to 60 minutes.
        If you want the resource to preferably avoid running on some nodes but
        be able to failover to them use 'pcs constraint location avoids'.

    ban <resource id> [node] [--master] [lifetime=<lifetime>] [--wait[=n]]
        Prevent the resource id specified from running on the node (or on the
        current node it is running on if no node is specified) by creating a
        -INFINITY location constraint.  If --master is used the scope of the
        command is limited to the master role and you must use the master id
        (instead of the resource id).  If lifetime is specified then the
        constraint will expire after that time, otherwise it defaults to
        infinity and the constraint can be cleared manually with 'pcs resource
        clear' or 'pcs constraint delete'.  If --wait is specified, pcs will
        wait up to 'n' seconds for the resource to move and then return 0
        on success or 1 on error. If 'n' is not specified it defaults to 60
        minutes.
        If you want the resource to preferably avoid running on some nodes but
        be able to failover to them use 'pcs constraint location avoids'.

    clear <resource id> [node] [--master] [--wait[=n]]
        Remove constraints created by move and/or ban on the specified
        resource (and node if specified).
        If --master is used the scope of the command is limited to the
        master role and you must use the master id (instead of the resource id).
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including starting and/or moving resources if
        appropriate) and then return 0 on success or 1 on error.  If 'n' is not
        specified it defaults to 60 minutes.

    standards
        List available resource agent standards supported by this installation
        (OCF, LSB, etc.).

    providers
        List available OCF resource agent providers.

    agents [standard[:provider]]
        List available agents optionally filtered by standard and provider.

    update <resource id> [resource options] [op [<operation action>
           <operation options>]...] [meta <meta operations>...] [--wait[=n]]
        Add/Change options to specified resource, clone or multi-state
        resource.  If an operation (op) is specified it will update the first
        found operation with the same action on the specified resource, if no
        operation with that action exists then a new operation will be created.
        (WARNING: all existing options on the updated operation will be reset
        if not specified.)  If you want to create multiple monitor operations
        you should use the 'op add' & 'op remove' commands.  If --wait is
        specified, pcs will wait up to 'n' seconds for the changes to take
        effect and then return 0 if the changes have been processed or 1
        otherwise.  If 'n' is not specified it defaults to 60 minutes.

    op add <resource id> <operation action> [operation properties]
        Add operation for specified resource.

    op remove <resource id> <operation action> [<operation properties>...]
        Remove specified operation (note: you must specify the exact operation
        properties to properly remove an existing operation).

    op remove <operation id>
        Remove the specified operation id.

    op defaults [options]
        Set default values for operations, if no options are passed, lists
        currently configured defaults. Defaults do not apply to resources which
        override them with their own defined operations.

    meta <resource id | group id | master id | clone id> <meta options>
         [--wait[=n]]
        Add specified options to the specified resource, group, master/slave
        or clone.  Meta options should be in the format of name=value, options
        may be removed by setting an option without a value.  If --wait is
        specified, pcs will wait up to 'n' seconds for the changes to take
        effect and then return 0 if the changes have been processed or 1
        otherwise.  If 'n' is not specified it defaults to 60 minutes.
        Example: pcs resource meta TestResource failure-timeout=50 stickiness=

    group add <group id> <resource id> [resource id] ... [resource id]
              [--before <resource id> | --after <resource id>] [--wait[=n]]
        Add the specified resource to the group, creating the group if it does
        not exist. If the resource is present in another group it is moved to
        the new group. You can use --before or --after to specify the position
        of the added resources relatively to some resource already existing in
        the group. By adding resources to a group they are already in and
        specifying --after or --before you can move the resources in the group.
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including moving resources if appropriate) and
        then return 0 on success or 1 on error. If 'n' is not specified it
        defaults to 60 minutes.

    group remove <group id> <resource id> [resource id] ... [resource id]
          [--wait[=n]]
        Remove the specified resource(s) from the group, removing the group if
        no resources remain in it. If --wait is specified, pcs will wait up to
        'n' seconds for the operation to finish (including moving resources if
        appropriate) and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    ungroup <group id> [resource id] ... [resource id] [--wait[=n]]
        Remove the group (note: this does not remove any resources from the
        cluster) or if resources are specified, remove the specified resources
        from the group.  If --wait is specified, pcs will wait up to 'n' seconds
        for the operation to finish (including moving resources if appropriate)
        and the return 0 on success or 1 on error.  If 'n' is not specified it
        defaults to 60 minutes.

    clone <resource id | group id> [clone options]... [--wait[=n]]
        Set up the specified resource or group as a clone. If --wait is
        specified, pcs will wait up to 'n' seconds for the operation to finish
        (including starting clone instances if appropriate) and then return 0
        on success or 1 on error. If 'n' is not specified it defaults to 60
        minutes.

    unclone <resource id | group id> [--wait[=n]]
        Remove the clone which contains the specified group or resource (the
        resource or group will not be removed).  If --wait is specified, pcs
        will wait up to 'n' seconds for the operation to finish (including
        stopping clone instances if appropriate) and then return 0 on success
        or 1 on error.  If 'n' is not specified it defaults to 60 minutes.

    master [<master/slave id>] <resource id | group id> [options] [--wait[=n]]
        Configure a resource or group as a multi-state (master/slave) resource.
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including starting and promoting resource
        instances if appropriate) and then return 0 on success or 1 on error.
        If 'n' is not specified it defaults to 60 minutes.
        Note: to remove a master you must remove the resource/group it contains.

    bundle create <bundle id> container <container type> [<container options>]
            [network <network options>] [port-map <port options>]...
            [storage-map <storage options>]... [meta <meta options>]
            [--disabled] [--wait[=n]]
        Create a new bundle encapsulating no resources. The bundle can be used
        either as it is or a resource may be put into it at any time.
        If --disabled is specified, the bundle is not started automatically.
        If --wait is specified, pcs will wait up to 'n' seconds for the bundle
        to start and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    bundle reset <bundle id> [container <container options>]
            [network <network options>] [port-map <port options>]...
            [storage-map <storage options>]... [meta <meta options>]
            [--disabled] [--wait[=n]]
        Configure specified bundle with given options. Unlike bundle update,
        this command resets the bundle according given options - no previous
        options are kept. Resources inside the bundle are kept as they are.
        If --disabled is specified, the bundle is not started automatically.
        If --wait is specified, pcs will wait up to 'n' seconds for the bundle
        to start and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    bundle update <bundle id> [container <container options>]
            [network <network options>]
            [port-map (add <port options>) | (remove <id>...)]...
            [storage-map (add <storage options>) | (remove <id>...)]...
            [meta <meta options>]
            [--wait[=n]]
        Add, remove or change options to specified bundle. If you wish to update
        a resource encapsulated in the bundle, use the 'pcs resource update'
        command instead and specify the resource id. If --wait is specified,
        pcs will wait up to 'n' seconds for the operation to finish (including
        moving resources if appropriate) and then return 0 on success or 1 on
        error. If 'n' is not specified it defaults to 60 minutes.

    manage <resource id>... [--monitor]
        Set resources listed to managed mode (default). If --monitor is
        specified, enable all monitor operations of the resources.

    unmanage <resource id>... [--monitor]
        Set resources listed to unmanaged mode. When a resource is in unmanaged
        mode, the cluster is not allowed to start nor stop the resource. If
        --monitor is specified, disable all monitor operations of the
        resources.

    defaults [options]
        Set default values for resources, if no options are passed, lists
        currently configured defaults. Defaults do not apply to resources which
        override them with their own defined values.

    cleanup [<resource id>] [--node <node>]
        Make the cluster forget failed operations from history of the resource
        and re-detect its current state. This can be useful to purge knowledge
        of past failures that have since been resolved. If a resource id is not
        specified then all resources / stonith devices will be cleaned up. If a
        node is not specified then resources / stonith devices on all nodes will
        be cleaned up.

    refresh [<resource id>] [--node <node>] [--full]
        Make the cluster forget the complete operation history (including
        failures) of the resource and re-detect its current state. If you are
        interested in forgetting failed operations only, use the 'pcs resource
        cleanup' command. If a resource id is not specified then all resources
        / stonith devices will be refreshed. If a node is not specified then
        resources / stonith devices on all nodes will be refreshed. Use --full
        to refresh a resource on all nodes, otherwise only nodes where the
        resource's state is known will be considered.

    failcount show [<resource id> [<node> [<operation> [<interval>]]]] [--full]
        Show current failcount for resources, optionally filtered by a resource,
        node, operation and its interval. If --full is specified do not sum
        failcounts per resource and node. Operation, interval and --full
        options require pacemaker-1.1.18 or newer.

    failcount reset [<resource id> [<node> [<operation> [interval=<interval>]]]]
        Reset failcount for specified resource on all nodes or only on specified
        node. This tells the cluster to forget how many times a resource has
        failed in the past. This may allow the resource to be started or moved
        to a more preferred location. Operation and interval options require
        pacemaker-1.1.18 or newer.

    relocate dry-run [resource1] [resource2] ...
        The same as 'relocate run' but has no effect on the cluster.

    relocate run [resource1] [resource2] ...
        Relocate specified resources to their preferred nodes.  If no resources
        are specified, relocate all resources.
        This command calculates the preferred node for each resource while
        ignoring resource stickiness.  Then it creates location constraints
        which will cause the resources to move to their preferred nodes.  Once
        the resources have been moved the constraints are deleted automatically.
        Note that the preferred node is calculated based on current cluster
        status, constraints, location of resources and other settings and thus
        it might change over time.

    relocate show
        Display current status of resources and their optimal node ignoring
        resource stickiness.

    relocate clear
        Remove all constraints created by the 'relocate run' command.

    utilization [<resource id> [<name>=<value> ...]]
        Add specified utilization options to specified resource. If resource is
        not specified, shows utilization of all resources. If utilization
        options are not specified, shows utilization of specified resource.
        Utilization option should be in format name=value, value has to be
        integer. Options may be removed by setting an option without a value.
        Example: pcs resource utilization TestResource cpu= ram=20

Examples:

    pcs resource show
      Show all resources.

    pcs resource show VirtualIP
      Show options specific to the 'VirtualIP' resource.


    pcs resource create VirtualIP ocf:heartbeat:IPaddr2 ip=192.168.0.99 \
               cidr_netmask=32 nic=eth2 op monitor interval=30s
      Create a new resource called 'VirtualIP' with options.

    pcs resource create VirtualIP IPaddr2 ip=192.168.0.99 \
               cidr_netmask=32 nic=eth2 op monitor interval=30s
      Create a new resource called 'VirtualIP' with options.

    pcs resource update VirtualIP ip=192.168.0.98 nic=
      Change the ip address of VirtualIP and remove the nic option.

    pcs resource delete VirtualIP
      Delete the VirtualIP resource.

Notes:
    Starting resources on a cluster is (almost) always done by pacemaker and
    not directly from pcs.  If your resource isn't starting, it's usually
    due to either a misconfiguration of the resource (which you debug in
    the system log), or constraints preventing the resource from starting or
    the resource being disabled.  You can use 'pcs resource debug-start' to
    test resource configuration, but it should *not* normally be used to start
    resources in a cluster.


[root@cluster1 ~]# pcs resource debug-start  halvm
Operation start for halvm (ocf:heartbeat:LVM) returned: 'unknown error' (1)
 >  stderr:   Volume group "vgcluster" not found
 >  stderr:   Cannot process volume group vgcluster
 >  stderr: ocf-exit-reason:Volume group [vgcluster] does not exist or contains error!   Volume group "vgcluster" not found
 >  stderr:   Cannot process volume group vgcluster
[root@cluster1 ~]# pcs resource create halvm LVM volgrpname=vgcluster exclusive=true --group halvmfs
Assumed agent name 'ocf:heartbeat:LVM' (deduced from 'LVM')
Error: 'halvm' already exists
[root@cluster1 ~]# Assumed agent name 'ocf:heartbeat:LVM' (deduced from 'LVM')
bash: syntax error near unexpected token `('
[root@cluster1 ~]# pcs resource remove halvm

Usage: pcs resource [commands]...
Manage pacemaker resources

Commands:
    [show [<resource id>] | --full | --groups | --hide-inactive]
        Show all currently configured resources or if a resource is specified
        show the options for the configured resource.  If --full is specified,
        all configured resource options will be displayed.  If --groups is
        specified, only show groups (and their resources).  If --hide-inactive
        is specified, only show active resources.

    list [filter] [--nodesc]
        Show list of all available resource agents (if filter is provided then
        only resource agents matching the filter will be shown). If --nodesc is
        used then descriptions of resource agents are not printed.

    describe [<standard>:[<provider>:]]<type> [--full]
        Show options for the specified resource. If --full is specified, all
        options including advanced ones are shown.

    create <resource id> [<standard>:[<provider>:]]<type> [resource options]
           [op <operation action> <operation options> [<operation action>
           <operation options>]...] [meta <meta options>...]
           [clone [<clone options>] | master [<master options>] |
           --group <group id> [--before <resource id> | --after <resource id>]
           | bundle <bundle id>] [--disabled] [--no-default-ops] [--wait[=n]]
        Create specified resource. If clone is used a clone resource is
        created. If master is specified a master/slave resource is created.
        If --group is specified the resource is added to the group named. You
        can use --before or --after to specify the position of the added
        resource relatively to some resource already existing in the group.
        If bundle is used, the resource will be created inside of the specified
        bundle. If --disabled is specified the resource is not started
        automatically. If --no-default-ops is specified, only monitor
        operations are created for the resource and all other operations use
        default settings. If --wait is specified, pcs will wait up to 'n'
        seconds for the resource to start and then return 0 if the resource is
        started, or 1 if the resource has not yet started. If 'n' is not
        specified it defaults to 60 minutes.
        Example: Create a new resource called 'VirtualIP' with IP address
            192.168.0.99, netmask of 32, monitored everything 30 seconds,
            on eth2:
            pcs resource create VirtualIP ocf:heartbeat:IPaddr2 \
                ip=192.168.0.99 cidr_netmask=32 nic=eth2 \
                op monitor interval=30s

    delete <resource id|group id|master id|clone id>
        Deletes the resource, group, master or clone (and all resources within
        the group/master/clone).

    enable <resource id>... [--wait[=n]]
        Allow the cluster to start the resources. Depending on the rest of the
        configuration (constraints, options, failures, etc), the resources may
        remain stopped. If --wait is specified, pcs will wait up to 'n' seconds
        for the resources to start and then return 0 if the resources are
        started, or 1 if the resources have not yet started. If 'n' is not
        specified it defaults to 60 minutes.

    disable <resource id>... [--wait[=n]]
        Attempt to stop the resources if they are running and forbid the
        cluster from starting them again. Depending on the rest of the
        configuration (constraints, options, failures, etc), the resources may
        remain started. If --wait is specified, pcs will wait up to 'n' seconds
        for the resources to stop and then return 0 if the resources are
        stopped or 1 if the resources have not stopped. If 'n' is not specified
        it defaults to 60 minutes.

    restart <resource id> [node] [--wait=n]
        Restart the resource specified. If a node is specified and if the
        resource is a clone or master/slave it will be restarted only on
        the node specified.  If --wait is specified, then we will wait
        up to 'n' seconds for the resource to be restarted and return 0 if
        the restart was successful or 1 if it was not.

    debug-start <resource id> [--full]
        This command will force the specified resource to start on this node
        ignoring the cluster recommendations and print the output from
        starting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to start.

    debug-stop <resource id> [--full]
        This command will force the specified resource to stop on this node
        ignoring the cluster recommendations and print the output from
        stopping the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to stop.

    debug-promote <resource id> [--full]
        This command will force the specified resource to be promoted on this
        node ignoring the cluster recommendations and print the output from
        promoting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to promote.

    debug-demote <resource id> [--full]
        This command will force the specified resource to be demoted on this
        node ignoring the cluster recommendations and print the output from
        demoting the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to demote.

    debug-monitor <resource id> [--full]
        This command will force the specified resource to be monitored on this
        node  ignoring the cluster recommendations and print the output from
        monitoring the resource.  Using --full will give more detailed output.
        This is mainly used for debugging resources that fail to be monitored.

    move <resource id> [destination node] [--master] [lifetime=<lifetime>]
         [--wait[=n]]
        Move the resource off the node it is currently running on by creating a
        -INFINITY location constraint to ban the node.  If destination node is
        specified the resource will be moved to that node by creating an
        INFINITY location constraint to prefer the destination node.  If
        --master is used the scope of the command is limited to the master role
        and you must use the master id (instead of the resource id).  If
        lifetime is specified then the constraint will expire after that time,
        otherwise it defaults to infinity and the constraint can be cleared
        manually with 'pcs resource clear' or 'pcs constraint delete'.  If
        --wait is specified, pcs will wait up to 'n' seconds for the resource
        to move and then return 0 on success or 1 on error.  If 'n' is not
        specified it defaults to 60 minutes.
        If you want the resource to preferably avoid running on some nodes but
        be able to failover to them use 'pcs constraint location avoids'.

    ban <resource id> [node] [--master] [lifetime=<lifetime>] [--wait[=n]]
        Prevent the resource id specified from running on the node (or on the
        current node it is running on if no node is specified) by creating a
        -INFINITY location constraint.  If --master is used the scope of the
        command is limited to the master role and you must use the master id
        (instead of the resource id).  If lifetime is specified then the
        constraint will expire after that time, otherwise it defaults to
        infinity and the constraint can be cleared manually with 'pcs resource
        clear' or 'pcs constraint delete'.  If --wait is specified, pcs will
        wait up to 'n' seconds for the resource to move and then return 0
        on success or 1 on error. If 'n' is not specified it defaults to 60
        minutes.
        If you want the resource to preferably avoid running on some nodes but
        be able to failover to them use 'pcs constraint location avoids'.

    clear <resource id> [node] [--master] [--wait[=n]]
        Remove constraints created by move and/or ban on the specified
        resource (and node if specified).
        If --master is used the scope of the command is limited to the
        master role and you must use the master id (instead of the resource id).
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including starting and/or moving resources if
        appropriate) and then return 0 on success or 1 on error.  If 'n' is not
        specified it defaults to 60 minutes.

    standards
        List available resource agent standards supported by this installation
        (OCF, LSB, etc.).

    providers
        List available OCF resource agent providers.

    agents [standard[:provider]]
        List available agents optionally filtered by standard and provider.

    update <resource id> [resource options] [op [<operation action>
           <operation options>]...] [meta <meta operations>...] [--wait[=n]]
        Add/Change options to specified resource, clone or multi-state
        resource.  If an operation (op) is specified it will update the first
        found operation with the same action on the specified resource, if no
        operation with that action exists then a new operation will be created.
        (WARNING: all existing options on the updated operation will be reset
        if not specified.)  If you want to create multiple monitor operations
        you should use the 'op add' & 'op remove' commands.  If --wait is
        specified, pcs will wait up to 'n' seconds for the changes to take
        effect and then return 0 if the changes have been processed or 1
        otherwise.  If 'n' is not specified it defaults to 60 minutes.

    op add <resource id> <operation action> [operation properties]
        Add operation for specified resource.

    op remove <resource id> <operation action> [<operation properties>...]
        Remove specified operation (note: you must specify the exact operation
        properties to properly remove an existing operation).

    op remove <operation id>
        Remove the specified operation id.

    op defaults [options]
        Set default values for operations, if no options are passed, lists
        currently configured defaults. Defaults do not apply to resources which
        override them with their own defined operations.

    meta <resource id | group id | master id | clone id> <meta options>
         [--wait[=n]]
        Add specified options to the specified resource, group, master/slave
        or clone.  Meta options should be in the format of name=value, options
        may be removed by setting an option without a value.  If --wait is
        specified, pcs will wait up to 'n' seconds for the changes to take
        effect and then return 0 if the changes have been processed or 1
        otherwise.  If 'n' is not specified it defaults to 60 minutes.
        Example: pcs resource meta TestResource failure-timeout=50 stickiness=

    group add <group id> <resource id> [resource id] ... [resource id]
              [--before <resource id> | --after <resource id>] [--wait[=n]]
        Add the specified resource to the group, creating the group if it does
        not exist. If the resource is present in another group it is moved to
        the new group. You can use --before or --after to specify the position
        of the added resources relatively to some resource already existing in
        the group. By adding resources to a group they are already in and
        specifying --after or --before you can move the resources in the group.
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including moving resources if appropriate) and
        then return 0 on success or 1 on error. If 'n' is not specified it
        defaults to 60 minutes.

    group remove <group id> <resource id> [resource id] ... [resource id]
          [--wait[=n]]
        Remove the specified resource(s) from the group, removing the group if
        no resources remain in it. If --wait is specified, pcs will wait up to
        'n' seconds for the operation to finish (including moving resources if
        appropriate) and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    ungroup <group id> [resource id] ... [resource id] [--wait[=n]]
        Remove the group (note: this does not remove any resources from the
        cluster) or if resources are specified, remove the specified resources
        from the group.  If --wait is specified, pcs will wait up to 'n' seconds
        for the operation to finish (including moving resources if appropriate)
        and the return 0 on success or 1 on error.  If 'n' is not specified it
        defaults to 60 minutes.

    clone <resource id | group id> [clone options]... [--wait[=n]]
        Set up the specified resource or group as a clone. If --wait is
        specified, pcs will wait up to 'n' seconds for the operation to finish
        (including starting clone instances if appropriate) and then return 0
        on success or 1 on error. If 'n' is not specified it defaults to 60
        minutes.

    unclone <resource id | group id> [--wait[=n]]
        Remove the clone which contains the specified group or resource (the
        resource or group will not be removed).  If --wait is specified, pcs
        will wait up to 'n' seconds for the operation to finish (including
        stopping clone instances if appropriate) and then return 0 on success
        or 1 on error.  If 'n' is not specified it defaults to 60 minutes.

    master [<master/slave id>] <resource id | group id> [options] [--wait[=n]]
        Configure a resource or group as a multi-state (master/slave) resource.
        If --wait is specified, pcs will wait up to 'n' seconds for the
        operation to finish (including starting and promoting resource
        instances if appropriate) and then return 0 on success or 1 on error.
        If 'n' is not specified it defaults to 60 minutes.
        Note: to remove a master you must remove the resource/group it contains.

    bundle create <bundle id> container <container type> [<container options>]
            [network <network options>] [port-map <port options>]...
            [storage-map <storage options>]... [meta <meta options>]
            [--disabled] [--wait[=n]]
        Create a new bundle encapsulating no resources. The bundle can be used
        either as it is or a resource may be put into it at any time.
        If --disabled is specified, the bundle is not started automatically.
        If --wait is specified, pcs will wait up to 'n' seconds for the bundle
        to start and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    bundle reset <bundle id> [container <container options>]
            [network <network options>] [port-map <port options>]...
            [storage-map <storage options>]... [meta <meta options>]
            [--disabled] [--wait[=n]]
        Configure specified bundle with given options. Unlike bundle update,
        this command resets the bundle according given options - no previous
        options are kept. Resources inside the bundle are kept as they are.
        If --disabled is specified, the bundle is not started automatically.
        If --wait is specified, pcs will wait up to 'n' seconds for the bundle
        to start and then return 0 on success or 1 on error. If 'n' is not
        specified it defaults to 60 minutes.

    bundle update <bundle id> [container <container options>]
            [network <network options>]
            [port-map (add <port options>) | (remove <id>...)]...
            [storage-map (add <storage options>) | (remove <id>...)]...
            [meta <meta options>]
            [--wait[=n]]
        Add, remove or change options to specified bundle. If you wish to update
        a resource encapsulated in the bundle, use the 'pcs resource update'
        command instead and specify the resource id. If --wait is specified,
        pcs will wait up to 'n' seconds for the operation to finish (including
        moving resources if appropriate) and then return 0 on success or 1 on
        error. If 'n' is not specified it defaults to 60 minutes.

    manage <resource id>... [--monitor]
        Set resources listed to managed mode (default). If --monitor is
        specified, enable all monitor operations of the resources.

    unmanage <resource id>... [--monitor]
        Set resources listed to unmanaged mode. When a resource is in unmanaged
        mode, the cluster is not allowed to start nor stop the resource. If
        --monitor is specified, disable all monitor operations of the
        resources.

    defaults [options]
        Set default values for resources, if no options are passed, lists
        currently configured defaults. Defaults do not apply to resources which
        override them with their own defined values.

    cleanup [<resource id>] [--node <node>]
        Make the cluster forget failed operations from history of the resource
        and re-detect its current state. This can be useful to purge knowledge
        of past failures that have since been resolved. If a resource id is not
        specified then all resources / stonith devices will be cleaned up. If a
        node is not specified then resources / stonith devices on all nodes will
        be cleaned up.

    refresh [<resource id>] [--node <node>] [--full]
        Make the cluster forget the complete operation history (including
        failures) of the resource and re-detect its current state. If you are
        interested in forgetting failed operations only, use the 'pcs resource
        cleanup' command. If a resource id is not specified then all resources
        / stonith devices will be refreshed. If a node is not specified then
        resources / stonith devices on all nodes will be refreshed. Use --full
        to refresh a resource on all nodes, otherwise only nodes where the
        resource's state is known will be considered.

    failcount show [<resource id> [<node> [<operation> [<interval>]]]] [--full]
        Show current failcount for resources, optionally filtered by a resource,
        node, operation and its interval. If --full is specified do not sum
        failcounts per resource and node. Operation, interval and --full
        options require pacemaker-1.1.18 or newer.

    failcount reset [<resource id> [<node> [<operation> [interval=<interval>]]]]
        Reset failcount for specified resource on all nodes or only on specified
        node. This tells the cluster to forget how many times a resource has
        failed in the past. This may allow the resource to be started or moved
        to a more preferred location. Operation and interval options require
        pacemaker-1.1.18 or newer.

    relocate dry-run [resource1] [resource2] ...
        The same as 'relocate run' but has no effect on the cluster.

    relocate run [resource1] [resource2] ...
        Relocate specified resources to their preferred nodes.  If no resources
        are specified, relocate all resources.
        This command calculates the preferred node for each resource while
        ignoring resource stickiness.  Then it creates location constraints
        which will cause the resources to move to their preferred nodes.  Once
        the resources have been moved the constraints are deleted automatically.
        Note that the preferred node is calculated based on current cluster
        status, constraints, location of resources and other settings and thus
        it might change over time.

    relocate show
        Display current status of resources and their optimal node ignoring
        resource stickiness.

    relocate clear
        Remove all constraints created by the 'relocate run' command.

    utilization [<resource id> [<name>=<value> ...]]
        Add specified utilization options to specified resource. If resource is
        not specified, shows utilization of all resources. If utilization
        options are not specified, shows utilization of specified resource.
        Utilization option should be in format name=value, value has to be
        integer. Options may be removed by setting an option without a value.
        Example: pcs resource utilization TestResource cpu= ram=20

Examples:

    pcs resource show
      Show all resources.

    pcs resource show VirtualIP
      Show options specific to the 'VirtualIP' resource.


    pcs resource create VirtualIP ocf:heartbeat:IPaddr2 ip=192.168.0.99 \
               cidr_netmask=32 nic=eth2 op monitor interval=30s
      Create a new resource called 'VirtualIP' with options.

    pcs resource create VirtualIP IPaddr2 ip=192.168.0.99 \
               cidr_netmask=32 nic=eth2 op monitor interval=30s
      Create a new resource called 'VirtualIP' with options.

    pcs resource update VirtualIP ip=192.168.0.98 nic=
      Change the ip address of VirtualIP and remove the nic option.

    pcs resource delete VirtualIP
      Delete the VirtualIP resource.

Notes:
    Starting resources on a cluster is (almost) always done by pacemaker and
    not directly from pcs.  If your resource isn't starting, it's usually
    due to either a misconfiguration of the resource (which you debug in
    the system log), or constraints preventing the resource from starting or
    the resource being disabled.  You can use 'pcs resource debug-start' to
    test resource configuration, but it should *not* normally be used to start
    resources in a cluster.


[root@cluster1 ~]# pcs resource delete halvm
Deleting Resource (and group) - halvm
[root@cluster1 ~]# pcs resource delete xfsfx
Error: Resource 'xfsfx' does not exist.
[root@cluster1 ~]# pcs resource delete xfsfs
Deleting Resource - xfsfs
[root@cluster1 ~]# pcs resource create halvm LVM volgrpname=vg exclusive=true --group halvmfs
Assumed agent name 'ocf:heartbeat:LVM' (deduced from 'LVM')
[root@cluster1 ~]# Assumed agent name 'ocf:heartbeat:LVM'
bash: Assumed: command not found...
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:06:03 2019
Last change: Sat Oct 26 23:05:44 2019 by root via cibadmin on cluster1

2 nodes configured
1 resource configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Resource Group: halvmfs
     halvm (ocf::heartbeat:LVM): Stopped

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs resource delete xfsfx
Error: Resource 'xfsfx' does not exist.
[root@cluster1 ~]# pcs resource delete halvm
Deleting Resource (and group) - halvm
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:06:18 2019
Last change: Sat Oct 26 23:06:15 2019 by root via cibadmin on cluster1

2 nodes configured
0 resources configured

Online: [ cluster1 cluster2 ]

No resources


Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs resource create halvm LVM volgrpname=vg exclusive=true --group halvmfs exclusive=true --group halvmfs
Error: --group can only be used once
[root@cluster1 ~]# Assumed agent name 'ocf:heartbeat:LVM'^C
[root@cluster1 ~]# pcs resource create halvm LVM volgrpname=vgcluster exclusive=true --group halvmfs  Assumed agent name 'ocf:heartbeat:LVM'
Error: missing value of 'Assumed' option
[root@cluster1 ~]# pcs resource create halvm LVM volgrpname=vgcluster exclusive=true --group halvmfs   agent name 'ocf:heartbeat:LVM'
Error: missing value of 'agent' option
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:08:45 2019
Last change: Sat Oct 26 23:06:15 2019 by root via cibadmin on cluster1

2 nodes configured
0 resources configured

Online: [ cluster1 cluster2 ]

No resources


Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs resource create my_lvm LVM volgrpname=vg exclusive=true --group apachegroup
Assumed agent name 'ocf:heartbeat:LVM' (deduced from 'LVM')
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:10:01 2019
Last change: Sat Oct 26 23:09:56 2019 by root via cibadmin on cluster1

2 nodes configured
1 resource configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Resource Group: apachegroup
     my_lvm (ocf::heartbeat:LVM): FAILED cluster2 (Monitoring)

Failed Resource Actions:
* my_lvm_monitor_0 on cluster2 'unknown error' (1): call=13, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=452ms
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:10:24 2019
Last change: Sat Oct 26 23:09:56 2019 by root via cibadmin on cluster1

2 nodes configured
1 resource configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Resource Group: apachegroup
     my_lvm (ocf::heartbeat:LVM): FAILED cluster2 (Monitoring)

Failed Resource Actions:
* my_lvm_monitor_0 on cluster2 'unknown error' (1): call=13, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=452ms
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs pcs resource create p_LVM ocf:heartbeat:LVM \
>   volgrpname=string \
>   op monitor depth="0" timeout="30" interval="10"

Usage: pcs [-f file] [-h] [commands]...
Control and configure pacemaker and corosync.

Options:
    -h, --help         Display usage and exit.
    -f file            Perform actions on file instead of active CIB.
    --debug            Print all network traffic and external commands run.
    --version          Print pcs version information. List pcs capabilities if
                       --full is specified.
    --request-timeout  Timeout for each outgoing request to another node in
                       seconds. Default is 60s.
    --force            Override checks and errors, the exact behavior depends on
                       the command. WARNING: Using the --force option is
                       strongly discouraged unless you know what you are doing.

Commands:
    cluster     Configure cluster options and nodes.
    resource    Manage cluster resources.
    stonith     Manage fence devices.
    constraint  Manage resource constraints.
    property    Manage pacemaker properties.
    acl         Manage pacemaker access control lists.
    qdevice     Manage quorum device provider on the local host.
    quorum      Manage cluster quorum settings.
    booth       Manage booth (cluster ticket manager).
    status      View cluster status.
    config      View and manage cluster configuration.
    pcsd        Manage pcs daemon.
    node        Manage cluster nodes.
    alert       Manage pacemaker alerts.
    client      Manage pcsd client configuration.

[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:11:52 2019
Last change: Sat Oct 26 23:09:56 2019 by root via cibadmin on cluster1

2 nodes configured
1 resource configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Resource Group: apachegroup
     my_lvm (ocf::heartbeat:LVM): FAILED cluster2 (Monitoring)

Failed Resource Actions:
* my_lvm_monitor_0 on cluster2 'unknown error' (1): call=13, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=452ms
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]#  pcs resource create p_LVM ocf:heartbeat:LVM   volgrpname=string   op monitor depth="0" timeout="30" interval="10"
Error: invalid resource operation option 'depth', allowed options are: OCF_CHECK_LEVEL, description, enabled, id, interval, interval-origin, name, on-fail, record-pending, requires, role, start-delay, timeout
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:12:18 2019
Last change: Sat Oct 26 23:09:56 2019 by root via cibadmin on cluster1

2 nodes configured
1 resource configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Resource Group: apachegroup
     my_lvm (ocf::heartbeat:LVM): FAILED cluster2 (Monitoring)

Failed Resource Actions:
* my_lvm_monitor_0 on cluster2 'unknown error' (1): call=13, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=452ms
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs resource delete my_lvm
Deleting Resource (and group) - my_lvm
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:12:38 2019
Last change: Sat Oct 26 23:12:35 2019 by root via cibadmin on cluster1

2 nodes configured
0 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 my_lvm (ocf::heartbeat:LVM): ORPHANED FAILED cluster2 (Monitoring)

Failed Resource Actions:
* my_lvm_monitor_0 on cluster2 'unknown error' (1): call=13, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=452ms
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# grep use_lvmetad /etc/lvm/lvm.conf
# See the use_lvmetad comment for a special case regarding filters.
#     This is incompatible with lvmetad. If use_lvmetad is enabled,
# Configuration option global/use_lvmetad.
# while use_lvmetad was disabled, it must be stopped, use_lvmetad
    use_lvmetad = 0
[root@cluster1 ~]# vi /etc/lvm/lvm.conf
[root@cluster1 ~]# vi /etc/lvm/lvm.conf
[root@cluster1 ~]# ps -ef | grep -i [l]vm
[root@cluster1 ~]#  vgs --noheadings -o vg_name
  centos
  vg   
[root@cluster1 ~]# pcs resource delete my_lvm
Error: Resource 'my_lvm' does not exist.
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:16:08 2019
Last change: Sat Oct 26 23:12:35 2019 by root via cibadmin on cluster1

2 nodes configured
0 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 my_lvm (ocf::heartbeat:LVM): ORPHANED FAILED cluster2 (Monitoring)

Failed Resource Actions:
* my_lvm_monitor_0 on cluster2 'unknown error' (1): call=13, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=452ms
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs cluster restart

Usage: pcs cluster [commands]...
Configure cluster for use with pacemaker

Commands:
    auth [<node>[:<port>]] [...] [-u <username>] [-p <password>] [--force]
            [--local]
        Authenticate pcs to pcsd on nodes specified, or on all nodes
        configured in the local cluster if no nodes are specified (authorization
        tokens are stored in ~/.pcs/tokens or /var/lib/pcsd/tokens for root).
        By default all nodes are also authenticated to each other, using
        --local only authenticates the local node (and does not authenticate
        the remote nodes with each other). Using --force forces
        re-authentication to occur.

    setup [--start [--wait[=<n>]]] [--local] [--enable] --name <cluster name>
            <node1[,node1-altaddr]> [<node2[,node2-altaddr]>] [...]
            [--transport udpu|udp] [--rrpmode active|passive]
            [--addr0 <addr/net> [[[--mcast0 <address>] [--mcastport0 <port>]
                            [--ttl0 <ttl>]] | [--broadcast0]]
            [--addr1 <addr/net> [[[--mcast1 <address>] [--mcastport1 <port>]
                            [--ttl1 <ttl>]] | [--broadcast1]]]]
            [--wait_for_all=<0|1>] [--auto_tie_breaker=<0|1>]
            [--last_man_standing=<0|1> [--last_man_standing_window=<time in ms>]]
            [--ipv6] [--token <timeout>] [--token_coefficient <timeout>]
            [--join <timeout>] [--consensus <timeout>] [--netmtu <size>]
            [--miss_count_const <count>] [--fail_recv_const <failures>]
            [--encryption 0|1]
        Configure corosync and sync configuration out to listed nodes.
        --local will only perform changes on the local node,
        --start will also start the cluster on the specified nodes,
        --wait will wait up to 'n' seconds for the nodes to start,
        --enable will enable corosync and pacemaker on node startup,
        --transport allows specification of corosync transport (default: udpu;
            udp for RHEL 6 clusters),
        --rrpmode allows you to set the RRP mode of the system. Currently only
            'passive' is supported or tested (using 'active' is not
            recommended).
        The --wait_for_all, --auto_tie_breaker, --last_man_standing,
            --last_man_standing_window options are all documented in corosync's
            votequorum(5) man page. These options are not supported on RHEL 6
            clusters.
        --ipv6 will configure corosync to use ipv6 (instead of ipv4). This
            option is not supported on RHEL 6 clusters.
        --token <timeout> sets time in milliseconds until a token loss is
            declared after not receiving a token (default 1000 ms;
            10000 ms for CMAN clusters)
        --token_coefficient <timeout> sets time in milliseconds used for
            clusters with at least 3 nodes as a coefficient for real token
            timeout calculation
            (token + (number_of_nodes - 2) * token_coefficient) (default 650 ms)
            This option is not supported on RHEL 6 clusters.
        --join <timeout> sets time in milliseconds to wait for join messages
            (default 50 ms)
        --consensus <timeout> sets time in milliseconds to wait for consensus
            to be achieved before starting a new round of membership
            configuration (default 1200 ms)
        --netmtu <size> sets the network maximum transmit unit (default: 1500)
        --miss_count_const <count> sets the maximum number of times on
            receipt of a token a message is checked for retransmission before
            a retransmission occurs (default 5 messages)
        --fail_recv_const <failures> specifies how many rotations of the token
            without receiving any messages when messages should be received
            may occur before a new configuration is formed
            (default 2500 failures)
        --encryption 0|1 disables (0) or enables (1) corosync communication
            encryption (default 0)

        Configuring Redundant Ring Protocol (RRP)

        When using udpu specifying nodes, specify the ring 0 address first
        followed by a ',' and then the ring 1 address.

        Example: pcs cluster setup --name cname nodeA-0,nodeA-1 nodeB-0,nodeB-1

        When using udp, using --addr0 and --addr1 will allow you to configure
        rrp mode for corosync.  It's recommended to use a network (instead of
        IP address) for --addr0 and --addr1 so the same corosync.conf file can
        be used around the cluster.  --mcast0 defaults to 239.255.1.1 and
        --mcast1 defaults to 239.255.2.1, --mcastport0/1 default to 5405 and
        ttl defaults to 1. If --broadcast is specified, --mcast0/1,
        --mcastport0/1 & --ttl0/1 are ignored.

    start [--all | <node>... ] [--wait[=<n>]] [--request-timeout=<seconds>]
        Start a cluster on specified node(s). If no nodes are specified then
        start a cluster on the local node. If --all is specified then start
        a cluster on all nodes. If the cluster has many nodes then the start
        request may time out. In that case you should consider setting
        --request-timeout to a suitable value. If --wait is specified, pcs
        waits up to 'n' seconds for the cluster to get ready to provide
        services after the cluster has successfully started.

    stop [--all | <node>... ] [--request-timeout=<seconds>]
        Stop a cluster on specified node(s). If no nodes are specified then
        stop a cluster on the local node. If --all is specified then stop
        a cluster on all nodes. If the cluster is running resources which take
        long time to stop then the stop request may time out before the cluster
        actually stops. In that case you should consider setting
        --request-timeout to a suitable value.

    kill
        Force corosync and pacemaker daemons to stop on the local node
        (performs kill -9). Note that init system (e.g. systemd) can detect that
        cluster is not running and start it again. If you want to stop cluster
        on a node, run pcs cluster stop on that node.

    enable [--all | <node>... ]
        Configure cluster to run on node boot on specified node(s). If node is
        not specified then cluster is enabled on the local node. If --all is
        specified then cluster is enabled on all nodes.

    disable [--all | <node>... ]
        Configure cluster to not run on node boot on specified node(s). If node
        is not specified then cluster is disabled on the local node. If --all
        is specified then cluster is disabled on all nodes.

    status
        View current cluster status (an alias of 'pcs status cluster').

    pcsd-status [<node>]...
        Show current status of pcsd on nodes specified, or on all nodes
        configured in the local cluster if no nodes are specified.

    sync
        Sync corosync configuration to all nodes found from current
        corosync.conf file (cluster.conf on systems running Corosync 1.x).

    cib [filename] [scope=<scope> | --config]
        Get the raw xml from the CIB (Cluster Information Base).  If a filename
        is provided, we save the CIB to that file, otherwise the CIB is
        printed.  Specify scope to get a specific section of the CIB.  Valid
        values of the scope are: configuration, nodes, resources, constraints,
        crm_config, rsc_defaults, op_defaults, status.  --config is the same as
        scope=configuration.  Do not specify a scope if you want to edit
        the saved CIB using pcs (pcs -f <command>).

    cib-push <filename> [--wait[=<n>]]
            [diff-against=<filename_original> | scope=<scope> | --config]
        Push the raw xml from <filename> to the CIB (Cluster Information Base).
        You can obtain the CIB by running the 'pcs cluster cib' command, which
        is recommended first step when you want to perform desired
        modifications (pcs -f <command>) for the one-off push.
        If diff-against is specified, pcs diffs contents of filename against
        contents of filename_original and pushes the result to the CIB.
        Specify scope to push a specific section of the CIB.  Valid values
        of the scope are: configuration, nodes, resources, constraints,
        crm_config, rsc_defaults, op_defaults.  --config is the same as
        scope=configuration.  Use of --config is recommended.  Do not specify
        a scope if you need to push the whole CIB or be warned in the case
        of outdated CIB.
        If --wait is specified wait up to 'n' seconds for changes to be applied.
        WARNING: the selected scope of the CIB will be overwritten by the
        current content of the specified file.
        Example:
            pcs cluster cib > original.xml
            cp original.xml new.xml
            pcs -f new.xml constraint location apache prefers node2
            pcs cluster cib-push new.xml diff-against=original.xml

    cib-upgrade
        Upgrade the CIB to conform to the latest version of the document schema.

    edit [scope=<scope> | --config]
        Edit the cib in the editor specified by the $EDITOR environment
        variable and push out any changes upon saving.  Specify scope to edit
        a specific section of the CIB.  Valid values of the scope are:
        configuration, nodes, resources, constraints, crm_config, rsc_defaults,
        op_defaults.  --config is the same as scope=configuration.  Use of
        --config is recommended.  Do not specify a scope if you need to edit
        the whole CIB or be warned in the case of outdated CIB.

    node add <node[,node-altaddr]> [--start [--wait[=<n>]]] [--enable]
            [--watchdog=<watchdog-path>] [--device=<path>] ...
            [--no-watchdog-validation]
        Add the node to the cluster and sync all relevant configuration files
        to the new node. If --start is specified also start cluster on the new
        node, if --wait is specified wait up to 'n' seconds for the new node to
        start. If --enable is specified configure cluster to start on the new
        node on boot.
        When using Redundant Ring Protocol (RRP) with udpu transport, specify
        the ring 0 address first followed by a ',' and then the ring 1 address.
        Use --watchdog to specify path to watchdog on newly added node, when
        SBD is enabled in cluster. If SBD is configured with shared storage,
        use --device to specify path to shared device on new node. If
        --no-watchdog-validation is specified, validation of watchdog will be
        skipped. This command can only be run on an existing cluster node.

        WARNING: By default, it is tested whether the specified watchdog is
                 supported. This may cause a restart of the system when
                 a watchdog with no-way-out-feature enabled is present. Use
                 --no-watchdog-validation to skip watchdog validation.

    node remove <node>
        Shutdown specified node and remove it from the cluster.

    node add-remote <node host> [<node name>] [options]
           [op <operation action> <operation options> [<operation action>
           <operation options>]...] [meta <meta options>...] [--wait[=<n>]]
        Add the node to the cluster as a remote node. Sync all relevant
        configuration files to the new node. Start the node and configure it to
        start the cluster on boot.
        Options are port and reconnect_interval. Operations and meta
        belong to an underlying connection resource (ocf:pacemaker:remote).
        If --wait is specified, wait up to 'n' seconds for the node to start.

    node remove-remote <node identifier>
        Shutdown specified remote node and remove it from the cluster.
        The node-identifier can be the name of the node or the address of the
        node.

    node add-guest <node host> <resource id> [options] [--wait[=<n>]]
        Make the specified resource a guest node resource. Sync all relevant
        configuration files to the new node. Start the node and configure it to
        start the cluster on boot.
        Options are remote-addr, remote-port and remote-connect-timeout.
        If --wait is specified, wait up to 'n' seconds for the node to start.

    node remove-guest <node identifier>
        Shutdown specified guest node and remove it from the cluster.
        The node-identifier can be the name of the node or the address of the
        node or id of the resource that is used as the guest node.

    node clear <node name>
        Remove specified node from various cluster caches. Use this if a
        removed node is still considered by the cluster to be a member of the
        cluster.

    uidgid
        List the current configured uids and gids of users allowed to connect
        to corosync.

    uidgid add [uid=<uid>] [gid=<gid>]
        Add the specified uid and/or gid to the list of users/groups
        allowed to connect to corosync.

    uidgid rm [uid=<uid>] [gid=<gid>]
        Remove the specified uid and/or gid from the list of users/groups
        allowed to connect to corosync.

    corosync [node]
        Get the corosync.conf from the specified node or from the current node
        if node not specified.

    reload corosync
        Reload the corosync configuration on the current node.

    destroy [--all]
        Permanently destroy the cluster on the current node, killing all
        cluster processes and removing all cluster configuration files. Using
        --all will attempt to destroy the cluster on all nodes in the local
        cluster.
        WARNING: This command permanently removes any cluster configuration that
        has been created. It is recommended to run 'pcs cluster stop' before
        destroying the cluster.

    verify [-V] [filename]
        Checks the pacemaker configuration (cib) for syntax and common
        conceptual errors.  If no filename is specified the check is
        performed on the currently running cluster.  If -V is used
        more verbose output will be printed.

    report [--from "YYYY-M-D H:M:S" [--to "YYYY-M-D H:M:S"]] <dest>
        Create a tarball containing everything needed when reporting cluster
        problems.  If --from and --to are not used, the report will include
        the past 24 hours.

[root@cluster1 ~]# pcs cluster stop
Stopping Cluster (pacemaker)...
Stopping Cluster (corosync)...
[root@cluster1 ~]# pcs cluster start
Starting Cluster (corosync)...
Starting Cluster (pacemaker)...
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: NONE
Last updated: Sat Oct 26 23:16:33 2019
Last change: Sat Oct 26 23:12:35 2019 by root via cibadmin on cluster1

2 nodes configured
0 resources configured

Node cluster1: UNCLEAN (offline)
Node cluster2: UNCLEAN (offline)

No resources


Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs cluster start
Starting Cluster (corosync)...
Starting Cluster (pacemaker)...
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster2 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sat Oct 26 23:16:47 2019
Last change: Sat Oct 26 23:12:35 2019 by root via cibadmin on cluster1

2 nodes configured
0 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 my_lvm (ocf::heartbeat:LVM): ORPHANED FAILED cluster2

Failed Resource Actions:
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# vi /etc/lvm/lvm.conf
[root@cluster1 ~]# pcs resource delete my_lvm
Error: Resource 'my_lvm' does not exist.
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sun Oct 27 05:36:47 2019
Last change: Sat Oct 26 23:12:35 2019 by root via cibadmin on cluster1

2 nodes configured
0 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 my_lvm (ocf::heartbeat:LVM): ORPHANED FAILED cluster2

Failed Resource Actions:
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pcs resource delete my_lvm
Error: Resource 'my_lvm' does not exist.
[root@cluster1 ~]# yum install lvm2-cluster gfs2-utils
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
epel/x86_64/metalink                                     | 5.4 kB     00:00   
 * base: mirrors.piconets.webwerks.in
 * epel: sg.fedora.ipserverone.com
 * extras: mirrors.piconets.webwerks.in
 * updates: mirrors.piconets.webwerks.in
base                                                     | 3.6 kB     00:00   
epel                                                     | 5.4 kB     00:00   
extras                                                   | 2.9 kB     00:00   
harbottle-main                                           | 3.3 kB     00:00   
updates                                                  | 2.9 kB     00:00   
(1/2): epel/x86_64/updateinfo                              | 1.0 MB   00:03   
(2/2): epel/x86_64/primary_db                              | 6.9 MB   00:25   
Package gfs2-utils-3.1.10-9.el7.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package lvm2-cluster.x86_64 7:2.02.185-2.el7 will be updated
---> Package lvm2-cluster.x86_64 7:2.02.185-2.el7_7.2 will be an update
--> Processing Dependency: lvm2 = 7:2.02.185-2.el7_7.2 for package: 7:lvm2-cluster-2.02.185-2.el7_7.2.x86_64
--> Running transaction check
---> Package lvm2.x86_64 7:2.02.185-2.el7 will be updated
---> Package lvm2.x86_64 7:2.02.185-2.el7_7.2 will be an update
--> Processing Dependency: lvm2-libs = 7:2.02.185-2.el7_7.2 for package: 7:lvm2-2.02.185-2.el7_7.2.x86_64
--> Running transaction check
---> Package lvm2-libs.x86_64 7:2.02.185-2.el7 will be updated
---> Package lvm2-libs.x86_64 7:2.02.185-2.el7_7.2 will be an update
--> Processing Dependency: device-mapper-event = 7:1.02.158-2.el7_7.2 for package: 7:lvm2-libs-2.02.185-2.el7_7.2.x86_64
--> Running transaction check
---> Package device-mapper-event.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-event.x86_64 7:1.02.158-2.el7_7.2 will be an update
--> Processing Dependency: device-mapper-event-libs = 7:1.02.158-2.el7_7.2 for package: 7:device-mapper-event-1.02.158-2.el7_7.2.x86_64
--> Processing Dependency: device-mapper = 7:1.02.158-2.el7_7.2 for package: 7:device-mapper-event-1.02.158-2.el7_7.2.x86_64
--> Running transaction check
---> Package device-mapper.x86_64 7:1.02.158-2.el7 will be updated
--> Processing Dependency: device-mapper = 7:1.02.158-2.el7 for package: 7:device-mapper-libs-1.02.158-2.el7.x86_64
---> Package device-mapper.x86_64 7:1.02.158-2.el7_7.2 will be an update
---> Package device-mapper-event-libs.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-event-libs.x86_64 7:1.02.158-2.el7_7.2 will be an update
--> Running transaction check
---> Package device-mapper-libs.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-libs.x86_64 7:1.02.158-2.el7_7.2 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                     Arch      Version                 Repository  Size
================================================================================
Updating:
 lvm2-cluster                x86_64    7:2.02.185-2.el7_7.2    updates    741 k
Updating for dependencies:
 device-mapper               x86_64    7:1.02.158-2.el7_7.2    updates    294 k
 device-mapper-event         x86_64    7:1.02.158-2.el7_7.2    updates    190 k
 device-mapper-event-libs    x86_64    7:1.02.158-2.el7_7.2    updates    189 k
 device-mapper-libs          x86_64    7:1.02.158-2.el7_7.2    updates    322 k
 lvm2                        x86_64    7:2.02.185-2.el7_7.2    updates    1.3 M
 lvm2-libs                   x86_64    7:2.02.185-2.el7_7.2    updates    1.1 M

Transaction Summary
================================================================================
Upgrade  1 Package (+6 Dependent packages)

Total size: 4.1 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : 7:device-mapper-1.02.158-2.el7_7.2.x86_64                   1/14
  Updating   : 7:device-mapper-libs-1.02.158-2.el7_7.2.x86_64              2/14
  Updating   : 7:device-mapper-event-libs-1.02.158-2.el7_7.2.x86_64        3/14
  Updating   : 7:device-mapper-event-1.02.158-2.el7_7.2.x86_64             4/14
  Updating   : 7:lvm2-libs-2.02.185-2.el7_7.2.x86_64                       5/14
  Updating   : 7:lvm2-2.02.185-2.el7_7.2.x86_64                            6/14
Failed to execute operation: Cannot send after transport endpoint shutdown
Failed to start lvm2-lvmetad.socket: Unit is masked.
  Updating   : 7:lvm2-cluster-2.02.185-2.el7_7.2.x86_64                    7/14
  Cleanup    : 7:lvm2-cluster-2.02.185-2.el7.x86_64                        8/14
  Cleanup    : 7:lvm2-2.02.185-2.el7.x86_64                                9/14
  Cleanup    : 7:lvm2-libs-2.02.185-2.el7.x86_64                          10/14
  Cleanup    : 7:device-mapper-event-1.02.158-2.el7.x86_64                11/14
  Cleanup    : 7:device-mapper-event-libs-1.02.158-2.el7.x86_64           12/14
  Cleanup    : 7:device-mapper-1.02.158-2.el7.x86_64                      13/14
  Cleanup    : 7:device-mapper-libs-1.02.158-2.el7.x86_64                 14/14
  Verifying  : 7:device-mapper-libs-1.02.158-2.el7_7.2.x86_64              1/14
  Verifying  : 7:lvm2-cluster-2.02.185-2.el7_7.2.x86_64                    2/14
  Verifying  : 7:device-mapper-event-libs-1.02.158-2.el7_7.2.x86_64        3/14
  Verifying  : 7:lvm2-2.02.185-2.el7_7.2.x86_64                            4/14
  Verifying  : 7:lvm2-libs-2.02.185-2.el7_7.2.x86_64                       5/14
  Verifying  : 7:device-mapper-event-1.02.158-2.el7_7.2.x86_64             6/14
  Verifying  : 7:device-mapper-1.02.158-2.el7_7.2.x86_64                   7/14
  Verifying  : 7:device-mapper-event-1.02.158-2.el7.x86_64                 8/14
  Verifying  : 7:device-mapper-1.02.158-2.el7.x86_64                       9/14
  Verifying  : 7:device-mapper-event-libs-1.02.158-2.el7.x86_64           10/14
  Verifying  : 7:lvm2-cluster-2.02.185-2.el7.x86_64                       11/14
  Verifying  : 7:device-mapper-libs-1.02.158-2.el7.x86_64                 12/14
  Verifying  : 7:lvm2-libs-2.02.185-2.el7.x86_64                          13/14
  Verifying  : 7:lvm2-2.02.185-2.el7.x86_64                               14/14

Updated:
  lvm2-cluster.x86_64 7:2.02.185-2.el7_7.2                                     

Dependency Updated:
  device-mapper.x86_64 7:1.02.158-2.el7_7.2                                   
  device-mapper-event.x86_64 7:1.02.158-2.el7_7.2                             
  device-mapper-event-libs.x86_64 7:1.02.158-2.el7_7.2                         
  device-mapper-libs.x86_64 7:1.02.158-2.el7_7.2                               
  lvm2.x86_64 7:2.02.185-2.el7_7.2                                             
  lvm2-libs.x86_64 7:2.02.185-2.el7_7.2                                       

Complete!
[root@cluster1 ~]#  pcs property set no-quorum-policy=freeze
[root@cluster1 ~]# pcs resource create dlm ocf:pacemaker:controld op monitor interval=30s on-fail=fence clone interleave=true ordered=true
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sun Oct 27 06:04:03 2019
Last change: Sun Oct 27 06:03:57 2019 by root via cibadmin on cluster1

2 nodes configured
2 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Clone Set: dlm-clone [dlm]
     Stopped: [ cluster1 cluster2 ]
 my_lvm (ocf::heartbeat:LVM): ORPHANED FAILED cluster2

Failed Resource Actions:
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# /sbin/lvmconf --enable-cluster
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sun Oct 27 06:04:42 2019
Last change: Sun Oct 27 06:03:57 2019 by root via cibadmin on cluster1

2 nodes configured
2 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Clone Set: dlm-clone [dlm]
     Stopped: [ cluster1 cluster2 ]
 my_lvm (ocf::heartbeat:LVM): ORPHANED FAILED cluster2

Failed Resource Actions:
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]#  pcs resource create clvmd ocf:heartbeat:clvm op monitor interval=30s on-fail=fence clone interleave=true ordered=true
[root@cluster1 ~]# pcs constraint order start dlm-clone then clvmd-clone
Adding dlm-clone clvmd-clone (kind: Mandatory) (Options: first-action=start then-action=start)
[root@cluster1 ~]# pcs constraint colocation add clvmd-clone with dlm-clone
[root@cluster1 ~]# pcs constraint colocation add clvmd-clone with dlm-clone^C
[root@cluster1 ~]#  pcs constraint colocation add clvmd-clone with dlm-clone
Error: duplicate constraint already exists, use --force to override
  clvmd-clone with dlm-clone (score:INFINITY) (id:colocation-clvmd-clone-dlm-clone-INFINITY)
[root@cluster1 ~]# s constraint order start dlm-clone then clvmd-clone
bash: s: command not found...
[root@cluster1 ~]# pcs constraint order start dlm-clone then clvmd-clone
Error: duplicate constraint already exists, use --force to override
  start dlm-clone then start clvmd-clone (kind:Mandatory) (id:order-dlm-clone-clvmd-clone-mandatory)
[root@cluster1 ~]#  pcs constraint colocation add clvmd-clone with dlm-clone --force
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sun Oct 27 06:07:18 2019
Last change: Sun Oct 27 06:07:13 2019 by root via cibadmin on cluster1

2 nodes configured
4 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Clone Set: dlm-clone [dlm]
     Stopped: [ cluster1 cluster2 ]
 Clone Set: clvmd-clone [clvmd]
     Stopped: [ cluster1 cluster2 ]
 my_lvm (ocf::heartbeat:LVM): ORPHANED FAILED cluster2

Failed Resource Actions:
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# pvcreate /dev/sdc
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Device /dev/sdc excluded by a filter.
[root@cluster1 ~]# fdisk /dev/sdc
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): n
Partition number (1-128, default 1):
First sector (34-20971486, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-20971486, default 20971486): +512MB
Created partition 1


Command (m for help): wq
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@cluster1 ~]# pvcreate /dev/sdc1
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
WARNING: ext4 signature detected on /dev/sdc1 at offset 1080. Wipe it? [y/n]: y
  Wiping ext4 signature on /dev/sdc1.
  Physical volume "/dev/sdc1" successfully created.
[root@cluster1 ~]# vgcreate -Ay -cy sasbin_vg  /dev/sdc1
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Clustered volume group "sasbin_vg" successfully created
[root@cluster1 ~]# lvcreate -L 450MB -n sasbin_lv sasbin_vg
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group sasbin_vg
[root@cluster1 ~]# mkfs.gfs2 -j2 -p lock_dlm -t rhel7-demo:sasbin /dev/sasbin_vg/sasbin_lv
/dev/sasbin_vg/sasbin_lv: No such file or directory
[root@cluster1 ~]# lvcreate -L 450M -n sasbin_lv sasbin_vg
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group sasbin_vg
[root@cluster1 ~]# vgcreate -Ay -cy sasbin_vg  /dev/sdc1
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  A volume group called sasbin_vg already exists.
[root@cluster1 ~]# pvcreate /dev/sdc1
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group sasbin_vg
  Device /dev/sdc1 excluded by a filter.
[root@cluster1 ~]# lvcreate -L 450M -n sasbin_lv sasbin_vg
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group sasbin_vg
[root@cluster1 ~]# mkfs.gfs2 -j2 -p lock_dlm -t rhel7-demo:sasbin /dev/sasbin_vg/sasbin_lv
/dev/sasbin_vg/sasbin_lv: No such file or directory
[root@cluster1 ~]#  lvcreate -L5G -n sasbin_lv sasbin_vg
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group sasbin_vg
[root@cluster1 ~]#  lvcreate -L .45G -n sasbin_lv sasbin_vg
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group sasbin_vg
[root@cluster1 ~]#  lvcreate -L5G -n sasbin_lv sasbin_vg
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group sasbin_vg
[root@cluster1 ~]# mkfs.gfs2 -j2 -p lock_dlm -t rhel7-demo:sasbin /dev/sasbin_vg/sasbin_lv
/dev/sasbin_vg/sasbin_lv: No such file or directory
[root@cluster1 ~]#  pcs property show
Cluster Properties:
 cluster-infrastructure: corosync
 cluster-name: cluster
 dc-version: 1.1.20-5.el7_7.1-3c4c782f70
 have-watchdog: false
 no-quorum-policy: freeze
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sun Oct 27 06:15:17 2019
Last change: Sun Oct 27 06:07:13 2019 by root via cibadmin on cluster1

2 nodes configured
4 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Clone Set: dlm-clone [dlm]
     Stopped: [ cluster1 cluster2 ]
 Clone Set: clvmd-clone [clvmd]
     Stopped: [ cluster1 cluster2 ]
 my_lvm (ocf::heartbeat:LVM): ORPHANED FAILED cluster2

Failed Resource Actions:
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# grep locking_type /etc/lvm/lvm.conf | egrep -v '#'
    locking_type = 3
[root@cluster1 ~]# vgcreate -Ay -cy --shared vgclvm /dev/sdc1
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group sasbin_vg
  Device /dev/sdc1 excluded by a filter.
[root@cluster1 ~]# fdisk /dev/sdc
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): n
Partition number (2-128, default 2):
First sector (34-20971486, default 1001472):
Last sector, +sectors or +size{K,M,G,T,P} (1001472-20971486, default 20971486): +1G
Created partition 2


Command (m for help): wq
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@cluster1 ~]# vgcreate -Ay -cy --shared vgclvm /dev/sdc1^C
[root@cluster1 ~]# pvcreate /dev/sdc2
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group sasbin_vg
  Physical volume "/dev/sdc2" successfully created.
[root@cluster1 ~]# vgcreate -Ay -cy --shared vgclvm /dev/sdc2
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group sasbin_vg
  Clustered volume group "vgclvm" successfully created
[root@cluster1 ~]# vgs
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
  Skipping clustered volume group sasbin_vg
  VG     #PV #LV #SN Attr   VSize   VFree
  centos   1   2   0 wz--n- <29.00g     0
  vg       1   1   0 wz--n- 484.00m 32.00m
[root@cluster1 ~]# lvcreate -l 100%FREE -n lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# mkfs.gfs2 -j3 -p lock_dlm -t mycluster:gfs2fs /dev/vgclvm/lvcluster
/dev/vgclvm/lvcluster: No such file or directory
[root@cluster1 ~]# lvcreate -l 1G -n lvcluster vgclvm
  Invalid argument for --extents: 1G
  Error during parsing of command line.
[root@cluster1 ~]# lvcreate -l 1GB -n lvcluster vgclvm
  Invalid argument for --extents: 1GB
  Error during parsing of command line.
[root@cluster1 ~]# lvcreate -L 1GB -n lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# lvcreate -L1GB -n lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# mkfs.gfs2 -j3 -p lock_dlm -t mycluster:gfs2fs /dev/vgclvm/lvcluster
/dev/vgclvm/lvcluster: No such file or directory
[root@cluster1 ~]# lvcreate -L1GB -n lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# lvcreate -l 100%FREE -n lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# lvcreate -l100%FREE -n lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# mkfs.gfs2 -j3 -p lock_dlm -t mycluster:gfs2fs /dev/vgclvm/lvcluster
/dev/vgclvm/lvcluster: No such file or directory
[root@cluster1 ~]#  /sbin/lvmconf --enable-cluster
[root@cluster1 ~]# mkfs.gfs2 -j3 -p lock_dlm -t mycluster:gfs2fs /dev/vgclvm/lvcluster
/dev/vgclvm/lvcluster: No such file or directory
[root@cluster1 ~]# lvs
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
  Skipping clustered volume group sasbin_vg
  LV   VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root centos -wi-ao---- <26.00g                                                   
  swap centos -wi-ao----   3.00g                                                   
  lv   vg     -wi------- 452.00m                                                   
[root@cluster1 ~]# history | grep lvcreate
  681  lvcreate -L 500M -n lv vg
  682  lvcreate -L 450M -n lv vg
  788  lvcreate -L 450MB -n sasbin_lv sasbin_vg
  790  lvcreate -L 450M -n sasbin_lv sasbin_vg
  793  lvcreate -L 450M -n sasbin_lv sasbin_vg
  795   lvcreate -L5G -n sasbin_lv sasbin_vg
  796   lvcreate -L .45G -n sasbin_lv sasbin_vg
  797   lvcreate -L5G -n sasbin_lv sasbin_vg
  807  lvcreate -l 100%FREE -n lvcluster vgclvm
  809  lvcreate -l 1G -n lvcluster vgclvm
  810  lvcreate -l 1GB -n lvcluster vgclvm
  811  lvcreate -L 1GB -n lvcluster vgclvm
  812  lvcreate -L1GB -n lvcluster vgclvm
  814  lvcreate -L1GB -n lvcluster vgclvm
  815  lvcreate -l 100%FREE -n lvcluster vgclvm
  816  lvcreate -l100%FREE -n lvcluster vgclvm
  821  history | grep lvcreate
[root@cluster1 ~]# lvcreate -L 1G -n  lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# mkfs.gfs2 -j3 -p lock_dlm -t mycluster:gfs2fs /dev/vgclvm/lvcluster
/dev/vgclvm/lvcluster: No such file or directory
[root@cluster1 ~]# systemctl disable lvm2-lvmetad --now
[root@cluster1 ~]# lvcreate -L 1G -n  lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# mkfs.gfs2 -j3 -p lock_dlm -t mycluster:gfs2fs /dev/vgclvm/lvcluster
/dev/vgclvm/lvcluster: No such file or directory
[root@cluster1 ~]# pcs status
Cluster name: cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Stack: corosync
Current DC: cluster1 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
Last updated: Sun Oct 27 06:28:33 2019
Last change: Sun Oct 27 06:07:13 2019 by root via cibadmin on cluster1

2 nodes configured
4 resources configured

Online: [ cluster1 cluster2 ]

Full list of resources:

 Clone Set: dlm-clone [dlm]
     Stopped: [ cluster1 cluster2 ]
 Clone Set: clvmd-clone [clvmd]
     Stopped: [ cluster1 cluster2 ]
 my_lvm (ocf::heartbeat:LVM): ORPHANED FAILED cluster2

Failed Resource Actions:
* my_lvm_monitor_10000 on cluster2 'unknown error' (1): call=14, status=complete, exitreason='The volume_list filter must be initialized in lvm.conf for exclusive activation without clvmd',
    last-rc-change='Sat Oct 26 23:09:56 2019', queued=0ms, exec=139ms

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
[root@cluster1 ~]# lvcreate -L 1G -n  lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# lvcreate -L .9G -n  lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# service clvmd status
Redirecting to /bin/systemctl status clvmd.service
Unit clvmd.service could not be found.
[root@cluster1 ~]# pcs constraint order start clvmd-clone then clusterfs-clone
Error: Resource 'clusterfs-clone' does not exist
[root@cluster1 ~]# pcs constraint order start dlm-clone then clvmd-clone
Error: duplicate constraint already exists, use --force to override
  start dlm-clone then start clvmd-clone (kind:Mandatory) (id:order-dlm-clone-clvmd-clone-mandatory)
[root@cluster1 ~]# pcs constraint order start dlm-clone then clvmd-clone --force
Adding dlm-clone clvmd-clone (kind: Mandatory) (Options: first-action=start then-action=start)
[root@cluster1 ~]# lvcreate -L .9G -n  lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# lvcreate -L 1G -n  lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# mkfs.gfs2 -j3 -p lock_dlm -t mycluster:gfs2fs /dev/vgclvm/lvcluster
/dev/vgclvm/lvcluster: No such file or directory
[root@cluster1 ~]# lvs
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
  Skipping clustered volume group sasbin_vg
  LV   VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root centos -wi-ao---- <26.00g                                                   
  swap centos -wi-ao----   3.00g                                                   
  lv   vg     -wi------- 452.00m                                                   
[root@cluster1 ~]# lvcreate -l 100%FREE -n lvcluster vgclvm
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  Skipping clustered volume group vgclvm
[root@cluster1 ~]# reb