All public logs

Combined display of all available logs of HPCWIKI. You can narrow down the view by selecting a log type, the username (case-sensitive), or the affected page (also case-sensitive).

• 12:47, 4 January 2024 Admin talk contribs created page Manual:SS System/Storage Pool (Created page with " ==Hot Spare automatic replacement== One of the key features of ZFS is detect failed hard disk and replace it with healthy spare drive automatically. The '''hot spare device''' means that the device(s) is not a part of active device in a pool, but if one of active device in the pool fails, the hot spare device(s) will replaces the failed device automatically. ===Add/Remove spare drive=== <syntaxhighlight lang="bash"> Add new hard drive(s) into storage pool #zpool add zss...") Tag: Visual edit
• 12:39, 4 January 2024 Admin talk contribs created page Manual:SS System/Network (Created page with " ==Network MTU size== To maximize data transfer throughput, zstorage has been set up with the largest MTU size (9000) so called as jumbo frame instead of general 1500. Overall data transaction speed would be enough with connect to a proper enterprise performance of 10G network switch.") Tag: Visual edit
• 12:37, 4 January 2024 Admin talk contribs created page Manual:SS System/Diskinfo (Created page with " == Disks in zstoreage (default Head + JBOD mode) == $ lsblk --scsi -o name,model,serial == Head LOG and Cache Configuration == Head node has two M.2 NVMe disk. The default zstorage has been configured one for OS (Samsung 970 EVO Plus 1TB) the other for ZFS Cache/LOG using device (Intel Optane 1TB). The Intel Optane parted with four partitions as 25% (238.5GB) of 1TB and plan to use LOG for zssd and zsas as well as cache for zssd and zsas. nvme1n1     259:0   ...") Tag: Visual edit
• 12:20, 4 January 2024 Admin talk contribs created page Manual:SS System (Created page with "== HPCMATE SS (Scalable Storage) System Manual == To provide scalable storage solution that can be tunable for various workload of storage, HPCMATE provides the best practice of ZFS environment storage system. This page describes how to apply pre-installed software and how to maintain the system. Like DLS system, all SS system based on HPCMATE system level optimized Ubuntu. == Type of SS system == Scalable storage system can be configured many combination. HPCMATE pr...") Tag: Visual edit
• 14:13, 3 January 2024 Admin talk contribs created page Private:DLS tuning (Created page with " == DLS network tuning == https://enterprise-support.nvidia.com/s/article/linux-sysctl-tuning <syntaxhighlight lang="bash"> #Increase number of connection net.core.somaxconn=2048 #Disable the TCP timestamps option for better CPU utilization: net.ipv4.tcp_timestamps=0 #Increase the maximum length of processor input queues: net.core.netdev_max_backlog=250000 #Enable the TCP selective acks option for better throughput: net.ipv4.tcp_sack=1 #how much of the socket buffer...") Tag: Visual edit
• 12:50, 3 January 2024 Admin talk contribs created page Vmstat (Created page with "The '''<code>vmstat</code>''' (virtual memory statistics) is a built-in monitoring utility in Linux. <ref>https://phoenixnap.com/kb/vmstat-command</ref> The command is used to obtain information about memory, system processes, paging, interrupts, block I/O, disk, and CPU scheduling. Users can observe system activity virtually in real-time by specifying a sampling period. == Output meaning == <syntaxhighlight lang="bash"> $ vmstat -n 1 procs -----------memory----------...") Tag: Visual edit
• 12:36, 3 January 2024 Admin talk contribs created page Linux kernel parameter (Created page with "== net.core.somaxconn == somaxconn is used to set the maximum number of connections that can be queued for a socket. This parameter is used to prevent a flood of connection requests from overwhelming the system. The maximum value for the <code>net.core.somaxconn</code> parameter depends on the system and kernel version. The default value for the <code>net.core.somaxconn</code> Linux kernel parameter is typically 128 and the maximum value is around 65535 on most systems...") Tag: Visual edit
• 12:28, 3 January 2024 Admin talk contribs moved page Kernel kernel param to Kernel boot param over redirect
• 12:28, 3 January 2024 Admin talk contribs deleted redirect Kernel boot param by overwriting (Deleted to make way for move from "Kernel kernel param")
• 09:33, 3 January 2024 Admin talk contribs created page File:InfiniBand Vs. RoCEv2.png
• 09:33, 3 January 2024 Admin talk contribs uploaded File:InfiniBand Vs. RoCEv2.png
• 09:01, 3 January 2024 Admin talk contribs created page File:RDMA FLOW.png
• 09:01, 3 January 2024 Admin talk contribs uploaded File:RDMA FLOW.png
• 08:48, 3 January 2024 Admin talk contribs moved page Verify that RDMA is working to RDMA
• 12:42, 2 January 2024 Admin talk contribs created page NFSv4 ACLs (Created page with "NFSv4 ACLs (Access Control Lists) are mechanism to manipulate access controls on network-mounted filesystems to supplement traditional Unix permissions.<ref>https://www.osc.edu/book/export/html/4523</ref> == Commands == {| class="wikitable" |+ ! ! ! |- |<code>nfs4_setfacl</code> |to add, remove, or modify the <abbr>ACL</abbr> | * <code>-a</code> – to '''add''' the specified Access Control Entry (ACE - defined below). Basically, this adds a new rule. * <code>-x</code>...") Tag: Visual edit
• 10:48, 2 January 2024 Admin talk contribs created page Mount vs bind mount (Created page with "In Linux systems, '''we can mount a device in a directory using the ''mount'' command'''. This allows us to access the device’s filesystem<syntaxhighlight lang="bash"> $ findmnt --real TARGET SOURCE FSTYPE OPTIONS / /dev/sda5 ext4 rw,relatime,errors=remount-ro ├─/sys/kernel/tracing tracefs tracefs rw,nosuid,nodev,noexec,relatime └─/boot/efi /dev/sda1 vfat rw,relatime,fmask=0077,dmask=0077,codepage=437,ioc...") Tag: Visual edit
• 10:27, 2 January 2024 Admin talk contribs created page SSHFS (Created page with "SSHFS (SSH Filesystem) is a filesystem client based on FUSE for mounting remote directories over an SSH connection. SSHFS is using the SFTP protocol, which is a subsystem of SSH and it is enabled by default on most SSH servers. When compared to other network file system protocols such as NFS and Samba the advantage of SSHFS is that it does not require any additional configuration on the server side. To use SSHFS you only need SSH access to the remote server via SFTP,...") Tag: Visual edit: Switched
• 09:53, 2 January 2024 Admin talk contribs created page NFS tips and tricks (Created page with " == nfsd: too many open connections, consider increasing the number of threads == NFS uses threads on the server to handle incoming and outgoing I/O requests. Using threads helps NFS scale to handle large numbers of clients and large numbers of I/O requests. dmesg display when reach some high number of clients. should increase Number of NFS daemons (''nfsd'') by changing the RPCNFSDCOUNT variable in <code>/etc/default/nfs-kernel-server</code> on a Debian-based machine....") Tag: Visual edit
• 09:04, 2 January 2024 Admin talk contribs created page Fstab (Created page with "In linux, fstab (aka, ''file systems table'') is a configuration table designed to mount and unmount file systems to a machine. The fstab file is read by the <code>mount</code> command, which happens automatically at boot time to determine the overall file system structure, and thereafter when a user executes the <code>mount</code> command to modify that structure == Fstab syntax == fstab is a 6 column structure, where each column designates a specific parameter and m...") Tag: Visual edit
• 16:01, 31 December 2023 Admin talk contribs moved page ZFS to Private:ZFS
• 15:55, 31 December 2023 Admin talk contribs created page Linux File System Types (Created page with "A file system on Linux is responsible for organizing and managing data on storage devices such as hard drives, solid-state drives (SSDs), and USB drives. Linux offers a diverse range of file system types, each tailored to specific use cases and scenarios. == Common Linux file system types == {| class="wikitable" |+ !Type !Description !Partition & mount |- |Ext4 |The most widely used and default file system. * High performance for most workloads. * Reliable journaling...") Tag: Visual edit
• 15:07, 31 December 2023 Admin talk contribs created page USB connector (Created page with "== USB 3.x Spec. == The naming convention of USB has become really messy. Here’s how it breaks down.<ref>https://www.cined.com/storage-explained-configure-raid-file-system-pros-cons/</ref> * USB 3.2 Gen 1, is USB 3.0. It has a maximum throughput of 5Gbps. This is also known as SuperSpeed USB. * USB 3.2 Gen 2, is USB 3.1. It has a maximum throughput of 10Gbps. This is also known as SuperSpeed USB 10Gbps. * USB 3.2 Gen 2×2, is USB 3.2. It has a maximum throughput of 20...") Tag: Visual edit
• 15:04, 31 December 2023 Admin talk contribs created page File:Type of USB connector .png
• 15:04, 31 December 2023 Admin talk contribs uploaded File:Type of USB connector .png
• 11:34, 29 December 2023 Admin talk contribs created page File:RAID table.png
• 11:34, 29 December 2023 Admin talk contribs uploaded File:RAID table.png
• 11:21, 29 December 2023 Admin talk contribs created page CacheCade drive (Created page with "LSI is offering CacheCade features that is similar to Adaptec's maxCache technology to to maximize transactional I/O performance through fast SSD as a cache device. CacheCade Pro 2.0 is available and only SSDs approved by LSI should be used together with CacheCade<ref>https://docs.broadcom.com/doc/12350215</ref> == CacheCade work flow == * One or more SSDs will be connected to the RAID controller in addition to the normal hard disks, when using CacheCade. * Thereby, t...") Tag: Visual edit
• 14:03, 28 December 2023 Admin talk contribs created page SED (Self-Encrypting Drives) (Created page with "A self-encrypting drive (SED) is a hard disk drive (HDD) or solid-state drive (SSD) designed to automatically encrypt and decrypt drive data without the need for user input or disk encryption software. Samsung, Seagate, and Toshiba, have SEDs on the market today. center|frameless|536x536px|SEDs works - Trusted Computing Group Many self-encrypting drives available today implement the OPAL and Enterprise standards developed b...") Tag: Visual edit
• 13:35, 28 December 2023 Admin talk contribs created page File:SEDs works-Trusted Computing Group.png
• 13:35, 28 December 2023 Admin talk contribs uploaded File:SEDs works-Trusted Computing Group.png
• 11:09, 23 December 2023 Admin talk contribs created page CheckInstall (Created page with "Have you ever want to build your own .deb package for distribution ? Ubuntu's checkinstall<ref>https://help.ubuntu.com/community/CheckInstall</ref> helps us to make custom .deb package. CheckInstall keeps track of all files installed by a "make install" or equivalent, creates a Slackware, RPM, or Debian package with those files, and adds it to the installed packages database, allowing for easy package removal or distribution. with checkinstall we can create Debian, RP...") Tag: Visual edit
• 09:43, 23 December 2023 Admin talk contribs created page Private:Inhouse compile/OpenSSL (Created page with "== OpenSSL == OpenSSL<ref>https://github.com/openssl</ref> is a software library for applications that secure communications over computer networks against eavesdropping or need to identify the party at the other end. It is widely used by Internet servers, including the majority of HTTPS websites.<ref>https://endoflife.date/openssl</ref> This guide intends to teach you to '''Install the Latest version of OpenSSL on Ubuntu 20.04'''. OpenSSL is an open-source command line...") Tag: Visual edit
• 09:32, 23 December 2023 Admin talk contribs created page RPATH (Created page with "'''rpath''' designates the run-time search path hard-coded in an executable file or library. Dynamic linking loaders use the rpath to find required libraries. <ref>https://en.wikipedia.org/wiki/Rpath</ref> When you are '''compiling''' a program, you create object files and then '''link''' them together. You may use GNU ld(1) to link them, there are also other linkers, LLVM linker. A linker combines object files into executable. After then when you '''execute''' an alre...") Tag: Visual edit
• 10:13, 21 December 2023 Admin talk contribs created page Setup nbconvert (Created page with "== nbconvert == The '''nbconvert''' tool<ref>https://github.com/jupyter/nbconvert</ref> (jupyter nbconvert) converts notebooks to various other formats via Jinja templates. The nbconvert tool allows you to convert an .ipynb into * HTML * LaTeX * PDF * Reveal JS * Markdown (md) * ReStructured Text (rst) * executable script * etc == Setup nvconvert on Ubuntu == <syntaxhighlight lang="bash"> #Install pandoc sudo apt-get install pandoc # install nbconvert with webpdf su...") Tag: Visual edit
• 14:00, 20 December 2023 Admin talk contribs deleted page Manual:DLSSystem/perl (content was: "== Perl in conda environment == Activate conda environment then install cpanm to manage further perl modules<syntaxhighlight lang="bash"> # install perl packages managemer cpanm inside conda environment mamba install perl-app-cpanminus # Using cpanm, we can install various perl module cpanm List::Util XML::Simple DBD::SQLite CGI.pm # perl jupyter connector cpanm Devel::IPerl </s...", and the only contributor was "Admin" (talk))
• 13:58, 20 December 2023 Admin talk contribs created page Manual:DLSSystem/perl (Created page with "== Perl in conda environment == Activate conda environment then install cpanm to manage further perl modules<syntaxhighlight lang="bash"> # install perl packages managemer cpanm inside conda environment mamba install perl-app-cpanminus # Using cpanm, we can install various perl module cpanm List::Util XML::Simple DBD::SQLite CGI.pm # perl jupyter connector cpanm Devel::IPerl </syntaxhighlight> == References == <references />") Tag: Visual edit
• 11:17, 20 December 2023 Admin talk contribs created page Manual:DLSsystem/network example (Created page with "== DLS Network bonding example == Pre-configured standalone DLS system shipped with link aggregation and the configuration file looks like following,<syntaxhighlight lang="bash"> $ cat /etc/netplan/01-netcfg.yaml # This file describes the network interfaces available on your system # For more information, see netplan(5). network: version: 2 renderer: networkd # example of DHCP # ethernets: # eno1: # dhcp4: yes # example of static # ethernets: #...") Tag: Visual edit
• 10:49, 20 December 2023 Admin talk contribs created page Manual:UCM/PAHEcontrol (Created page with " == Control PA/HE on Master == PA and HE can be controlled from master node<syntaxhighlight lang="bash"> pahecontrol service is at /etc/systemd/system/pahecontrol.service which actually reference pahecontrol.py and config.json.schedule at /opt/hpcmate/cux $sudo systemctl disable pahecontrol.service $sudo systemctl enable pahecontrol.service $sudo systemctl start pahecontrol.service $sudo systemctl stop pahecontrol.service $sudo systemctl status pahecontrol.service </...") Tag: Visual edit
• 08:16, 20 December 2023 Admin talk contribs created page Conda vs Mamba comparision (Created page with "{| class="wikitable" |+ !Type !Mamba !Conda !Notes |- |clone env |Export the contents of the existing environment to a YAML file. <code>micromamba env export -n oldenv > oldenv.yaml</code> Create a new environment from the YAML file. <code>micromamba env create -n newenv -f oldenv.yaml</code> |conda create --name CLONE_ENV_NAME --clone ENV_NAME | |- | | | | |- | | | | |} == References == <references />") Tag: Visual edit
• 08:26, 19 December 2023 Admin talk contribs created page Manual:DLSSystem/dlsmodules (Created page with "== Preset modules and libraries == HPCMATE DLS has two type of systems - standalone and UCM cluster version Following table is a summary of preconfigured DLS modules {| class="wikitable" |+ !Module !Version !DLS !DLSBI !UCM clients !UCM master !Path |- |Polyphen |v2 | |O | | |/opt/hpcmate/tools/libs/pph2 |- |ncbi-blast |v2.15.0, v2.9.0 | |O | | |/opt/hpcmate/tools/libs/blast+-{version} |- | | | | | | | |} == References == <references />") Tag: Visual edit
• 07:41, 19 December 2023 Admin talk contribs created page Manual:DLSSystem/Conda configuration (Created page with " == HMDLS_PATH == $HMDLS_PATH activates when user ativate pre-setting dls or dlsbi environment and deactivate. $HMDLS_PATH variable will be added to default conda $PATH on bash environment and be removed on deactivation. <ref>https://stackoverflow.com/questions/63721599/conda-activate-d-and-deactivate-d-do-not-behave-what-i-expect</ref> <syntaxhighlight lang="bash"> # activate environment script is at $CONDA_PREFIX/etc/conda/activate.d/activate-hmdls.sh #!/bin/bash ex...") Tag: Visual edit
• 15:04, 17 December 2023 Admin talk contribs created page HMG493-SB1 (Created page with "center|frameless|726x726px|HMG493-SB1 ==Application[edit | edit source]== *Artificial intelligence *Heavy scientific computation *High-performance computing *Research lab/national lab *Astrophysics, business intelligence ==Features[edit | edit source]== #2 Intel Gen 4^th Sockets, 10-GPUs, 4U Accelerator Server for HPC and Storage # 4U formfactor, 10 double-width full-height GPUs over PCIe 5.0, Dual ROM Architecture # Dual processor, LGA...") Tag: Visual edit: Switched
• 14:51, 17 December 2023 Admin talk contribs created page File:HMG493-SB1.png
• 14:51, 17 December 2023 Admin talk contribs uploaded File:HMG493-SB1.png
• 10:48, 17 December 2023 Admin talk contribs created page Compile tips and tricks (Created page with "== GSL-config not found == <code>sudo apt install libgsl-dev</code><ref>https://askubuntu.com/questions/913493/gsl-config-not-found</ref> == References == <references />") Tag: Visual edit
• 13:54, 16 December 2023 Admin talk contribs created page File:Cable connector pinout2.png
• 13:54, 16 December 2023 Admin talk contribs uploaded File:Cable connector pinout2.png
• 13:52, 16 December 2023 Admin talk contribs deleted page File:Cable connector pinout.png (replace image)
• 13:38, 16 December 2023 Admin talk contribs created page File:EPS12 4+4.png
• 13:38, 16 December 2023 Admin talk contribs uploaded File:EPS12 4+4.png