Lewis Consultancy Wiki - User contributions [en]

Category:Test

2021-11-08T14:37:04Z

Clewis: Created page with "The Test category"

The Test category

Testpage2

2021-11-08T14:30:33Z

Clewis: Created page with "Another test page Category:Test"

Another test page

[[Category:Test]]

Testpage

2021-11-08T14:29:51Z

Clewis:

This is a test page

[[Category:Test]]

Testpage

2021-11-08T14:29:35Z

Clewis: Created page with "This is a test page Category: Test"

This is a test page

[[Category: Test]]

IT/AWS/S3

2021-09-01T09:01:15Z

Clewis: /* Tag files ( set tag archive=yes ) */

=Tag objects in an S3 bucket based on size=
It's desirable to avoid archiving small files as not efficient. One approach is to tag the file based on then archive to glacier based on tag.

==Get full filelist of bucket and store==
aws s3 ls s3://bucketname/path --recursive > /tmp/fullfilelist.txt

==Select only files greater than x bytes==
cat /tmp/fullfilelist.txt | awk '{ if ($3> 5000000) print $4}' > /tmp/largefilenames.txt

==Tag files ( set tag archive=yes ) ==
cat /tmp/largefilenames.txt | while read fn; do aws s3api put-object-tagging --bucket bucketname --key "$fn" --tagging '{"TagSet": [{ "Key": "archive", "Value": "yes" }]}'; done

IT/AWS/S3

2021-09-01T09:00:48Z

Clewis: /* Tag files (set tag archive=yes */

=Tag objects in an S3 bucket based on size=
It's desirable to avoid archiving small files as not efficient. One approach is to tag the file based on then archive to glacier based on tag.

==Get full filelist of bucket and store==
aws s3 ls s3://bucketname/path --recursive > /tmp/fullfilelist.txt

==Select only files greater than x bytes==
cat /tmp/fullfilelist.txt | awk '{ if ($3> 5000000) print $4}' > /tmp/largefilenames.txt

==Tag files ( set tag archive=yes ) ==
cat /tmp/largefilenames.txt | while read fn; do aws s3api put-object-tagging --bucket inview-nas1 --key "$fn" --tagging '{"TagSet": [{ "Key": "archive", "Value": "yes" }]}'; done

IT/AWS/S3

2021-09-01T09:00:32Z

Clewis: Created page with "=Tag objects in an S3 bucket based on size= It's desirable to avoid archiving small files as not efficient. One approach is to tag the file based on then archive to glacier ba..."

=Tag objects in an S3 bucket based on size=
It's desirable to avoid archiving small files as not efficient. One approach is to tag the file based on then archive to glacier based on tag.

==Get full filelist of bucket and store==
aws s3 ls s3://bucketname/path --recursive > /tmp/fullfilelist.txt

==Select only files greater than x bytes==
cat /tmp/fullfilelist.txt | awk '{ if ($3> 5000000) print $4}' > /tmp/largefilenames.txt

==Tag files (set tag archive=yes ==
cat /tmp/largefilenames.txt | while read fn; do aws s3api put-object-tagging --bucket inview-nas1 --key "$fn" --tagging '{"TagSet": [{ "Key": "archive", "Value": "yes" }]}'; done

Main Page/IT/Misc

2018-12-07T16:26:22Z

Clewis:

=Save tracks on a USB stick so that they play in order on some inferior Pioneer Car Stereo equipment=

First copy all contact to a temporary location ... let's say /var/tracks. Make sure all tracks are numbered in such a way that that they list in correct order numerically i.e. prefix with 01..02..//..10..11 etc

The destination location for this example is /media/stick/

<pre>find /var/tracks/ | sort -n | while read fn; do cp --parents -f "$fn" /media/usb-stick/; sleep 1; done</pre>

=Convert .mov file to h264 (mp4 container)=

Useful for Canon EOS cameras to save space on storage.

<pre>avconv -nostdin -i inputfile.mov -c:v libx264 outputfile.mp4</pre>

=Convert MTS to mkv and reduce scale=
<pre>ffmpeg -nostdin -i $fn -vf scale=1024:-1 ~/Videos/conv/$fn.mkv</pre>

or with deinterlacing

<pre> ffmpeg -i 00168.MTS -s 1024x768 -vcodec libx264 -crf 20 -acodec ac3 -vf "yadif" /mnt/mp4vids/00168.mp4</pre>

=Copy files found between two dates from a heirachical structure to a flat structure=
<pre>find . -type f -newermt 2016-08-07 ! -newermt 2016-08-30 -printf %p\|%f\\n | while read names
do IFS='|' read var1 var2 <<< `echo $names`; rsync -Pva "$var1" root@remote.machine.com:"/var/path/$var2"; done
</pre>

=Mount a Hauwei mobile phone=
Huawei seem to have a usb mass storage device which appears to allow ease of installing windows drivers. This stops the underlying mtp device from working in nautilus. It is possible to mount the mtp resources using jmtpfs :

sudo jmtpfs /mnt/phone -o allow_other

Main Page/IT/Misc

2018-09-09T10:36:40Z

Clewis:

Sandbox

2017-07-13T14:44:22Z

Clewis: Created page with "<span class="newwin">MediaWiki</span>"

<span class="newwin">[[MediaWiki]]</span>

MediaWiki:Common.js

2017-07-13T14:42:34Z

Clewis: Created page with "/* Any JavaScript here will be loaded for all users on every page load. */ $(function( $ ) { $( '#mw-content-text' ).on( 'click', '.newwin > a', function( e ) { var other..."

/* Any JavaScript here will be loaded for all users on every page load. */

$(function( $ ) {
$( '#mw-content-text' ).on( 'click', '.newwin > a', function( e ) {
var otherWindow = window.open();
otherWindow.opener = null;
otherWindow.location = this;
return false;
} );
} );

Main Page/IT/Misc

2017-02-26T11:20:05Z

Clewis:

=Save tracks on a USB stick so that they play in order on some inferior Pioneer Car Stereo equipment=

First copy all contact to a temporary location ... let's say /var/tracks. Make sure all tracks are numbered in such a way that that they list in correct order numerically i.e. prefix with 01..02..//..10..11 etc

The destination location for this example is /media/stick/

<pre>find /var/tracks/ | sort -n | while read fn; do cp --parents -f "$fn" /media/usb-stick/; sleep 1; done</pre>

=Convert .mov file to h264 (mp4 container)=

Useful for Canon EOS cameras to save space on storage.

<pre>avconv -i inputfile.mov -c:v libx264 outputfile.mp4</pre>

=Convert MTS to mkv and reduce scale=
<pre>ffmpeg -nostdin -i $fn -vf scale=1024:-1 ~/Videos/conv/$fn.mkv</pre>

or with deinterlacing

<pre> ffmpeg -i 00168.MTS -s 1024x768 -vcodec libx264 -crf 20 -acodec ac3 -vf "yadif" /mnt/mp4vids/00168.mp4</pre>

=Copy files found between two dates from a heirachical structure to a flat structure=
<pre>find . -type f -newermt 2016-08-07 ! -newermt 2016-08-30 -printf %p\|%f\\n | while read names
do IFS='|' read var1 var2 <<< `echo $names`; rsync -Pva "$var1" root@remote.machine.com:"/var/path/$var2"; done
</pre>

Main Page/IT/iDRAC

2016-09-01T10:37:31Z

Clewis: Created page with "=Some notes on iDRAC= ==Change the port number== Use <pre>racadm set iDRAC.Webserver.HttpsPort 445</pre> The racadm command is part of the dell omsa utilities."

=Some notes on iDRAC=

==Change the port number==
Use

<pre>racadm set iDRAC.Webserver.HttpsPort 445</pre>

The racadm command is part of the dell omsa utilities.

Main Page/IT

2016-09-01T10:36:57Z

Clewis:

=IT=
==Topics==
===[[/Admin]]===
===[[/bind]]===
===[[/clamav]]===
===[[/iDRAC]]===
===[[/LDAP]]===
===[[/LibreOffice]]===
===[[/Misc]]===
===[[/Postgresql]]===
===[[/ssl]]===
===[[/text processing]]===
===[[/Virtualisation]]===
===[[/ssh]]===
===[[/nagios]]===

Main Page/IT/Misc

2016-06-21T21:47:46Z

Clewis: /* Convert MTS to mkv and reduce scale */

Main Page/IT/nagios

2016-05-25T10:44:46Z

Clewis: Created page with "=A one liner to split up a nagios log= <pre>cat nagios.log | while read line; do unixts=`echo $line | cut -b 2-11`; logname="nagios-`date -d \@$unixts +%m-%d-%Y`-00.log"; e..."

=A one liner to split up a nagios log=
<pre>cat nagios.log | while read line; do unixts=`echo $line | cut -b 2-11`; logname="nagios-`date -d \@$unixts +%m-%d-%Y`-00.log"; echo $line >> $logname; done</pre>

Main Page/IT

2016-05-25T10:44:22Z

Clewis: /* Topics */

=IT=
==Topics==
===[[/Admin]]===
===[[/bind]]===
===[[/clamav]]===
===[[/LDAP]]===
===[[/LibreOffice]]===
===[[/Misc]]===
===[[/Postgresql]]===
===[[/ssl]]===
===[[/text processing]]===
===[[/Virtualisation]]===
===[[/ssh]]===
===[[/nagios]]===

Main Page/IT/LibreOffice

2016-02-09T15:26:36Z

Clewis: Created page with "=Remove Text Formatting Quote= Sometimes Libreoffice does bad things when you import CSV files. You can remove a single quote ' at the beginning of text formatted cells using ..."

=Remove Text Formatting Quote=
Sometimes Libreoffice does bad things when you import CSV files. You can remove a single quote ' at the beginning of text formatted cells using find and replace:
* select use regex option in find and replace
* Find for <pre> ^.*$ </pre>
* replace with <pre>& </pre>

Main Page/IT

2016-02-09T15:24:21Z

Clewis: /* Topics */

Main Page/IT/Postgresql

2016-02-04T12:00:25Z

Clewis:

===Back up a database remotely with compression===
<pre>ssh root@dbserver 'sudo -Hu postgres /usr/lib/postgresql/9.4/bin/pg_dumpall | gzip -f9' > output.sql.gz</pre>

Main Page/IT/Postgresql

2016-02-04T11:42:06Z

Clewis: Created page with "===Back up a database remotely with compression=== <pre>ssh root@dbserver 'sudo -Hu postgres /usr/lib/postgresql/9.4/bin/pg_dump dbname | gzip -f9' > output.sql.gz</pre>"

===Back up a database remotely with compression===
<pre>ssh root@dbserver 'sudo -Hu postgres /usr/lib/postgresql/9.4/bin/pg_dump dbname | gzip -f9' > output.sql.gz</pre>

Main Page/IT

2016-02-04T11:40:54Z

Clewis:

=IT=
==Topics==
===[[/Admin]]===
===[[/bind]]===
===[[/clamav]]===
===[[/LDAP]]===
===[[/Misc]]===
===[[/Postgresql]]===
===[[/ssl]]===
===[[/text processing]]===
===[[/Virtualisation]]===
===[[/ssh]]===

Main Page/IT/ssh

2015-11-04T14:53:16Z

Clewis:

=Set up a revserse tunnel to bypass NAT to a remote machine=
Useful to log into a remote machine (remote) with account (remoteaccount) that is behind a NAT firewall.

Create an account on a machine on the Internet (middle.some.place) which is available from both sides and set a password:
<pre>useradd -s /bin/bash middleuser
passwd middleuser</pre>

From remote machine
<pre>ssh -R 10002:localhost:22 middleuser@middle.some.place
</pre>
The remote user will log into the middle machine. The terminal must stay open for the duration of the connection.

if you're logged into the middle.some.place.... you can then do:

<pre> ssh remoteaccount@localhost -P 10002
</pre>

should present a login for the remote machine and that should be it!

Main Page/IT/ssh

2015-11-04T14:52:42Z

Clewis: Created page with "=Set up a revserse tunnel to bypass NAT to a remote machine= Useful to log into a remote machine (remote) with account (remoteaccount) that is behind a NAT firewall. Create a..."

Main Page/IT

2015-11-04T14:42:04Z

Clewis: /* Topics */

=IT=
==Topics==
===[[/Admin]]===
===[[/bind]]===
===[[/clamav]]===
===[[/LDAP]]===
===[[/Misc]]===
===[[/ssl]]===
===[[/text processing]]===
===[[/Virtualisation]]===
===[[/ssh]]===

Main Page/IT/Misc

2015-07-20T22:40:54Z

Clewis: /* Convert .mov file to h264 (mp4 container) */

Main Page/IT/Misc

2015-01-03T23:08:03Z

Clewis:

Main Page/IT/Misc

2014-11-13T11:27:14Z

Clewis:

Main Page/IT/Misc

2014-11-13T11:26:31Z

Clewis: Created page with "=Save tracks on a USB stick so that they play in order on some inferior Pioneer Car Stereo equipment= First copy all contact to a temporary location ... let's say /var/tracks..."

Main Page/IT

2014-11-13T11:22:34Z

Clewis: /* Topics */

=IT=
==Topics==
===[[/Admin]]===
===[[/bind]]===
===[[/clamav]]===
===[[/LDAP]]===
===[[/Misc]]===
===[[/ssl]]===
===[[/text processing]]===
===[[/Virtualisation]]===

Main Page/IT/Admin

2014-10-22T15:00:01Z

Clewis: /* Which processes are listening on which ports? */

=Which processes are listening on which ports?=
<pre>netstat -tulpn</pre>
=Which processes are using all the IO=
<pre>iotop --only</pre>
The --only shows only processes using io as opposed to all processes

Main Page/IT/text processing/awk

2014-10-16T15:43:05Z

Clewis: Created page with "=print a column= <pre>ping $(ip route |awk '/default/{print $3}')</pre> Pings the default gateway as found from ip route."

=print a column=
<pre>ping $(ip route |awk '/default/{print $3}')</pre>
Pings the default gateway as found from ip route.

Main Page/IT/text processing

2014-10-16T15:41:21Z

Clewis:

===[[/awk]]===
===[[/grep]]===

Main Page/IT/Admin

2014-09-11T09:26:59Z

Clewis: Created page with "=Which processes are listening on which ports?= <pre>netstat -tulpn</pre>"

=Which processes are listening on which ports?=
<pre>netstat -tulpn</pre>

Main Page/IT

2014-09-11T09:26:25Z

Clewis:

=IT=
==Topics==
===[[/Admin]]===
===[[/Virtualisation]]===
===[[/LDAP]]===
===[[/bind]]===
===[[/ssl]]===
===[[/clamav]]===
===[[/text processing]]===

Main Page/IT/text processing/grep

2014-07-03T09:46:38Z

Clewis: Created page with "=grep= Some useful things to know ==Finding IPs only== <pre>cat filewithips | grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' | sort | uniq</pre>"

=grep=
Some useful things to know
==Finding IPs only==
<pre>cat filewithips | grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' | sort | uniq</pre>

Main Page/IT/text processing

2014-07-03T09:45:09Z

Clewis: Created page with "===/grep==="

===[[/grep]]===

Main Page/IT

2014-07-03T09:44:20Z

Clewis:

=IT=
==Topics==
===[[/Virtualisation]]===
===[[/LDAP]]===
===[[/bind]]===
===[[/ssl]]===
===[[/clamav]]===
===[[/text processing]]===

Main Page/IT

2014-07-03T09:44:00Z

Clewis:

=IT=
==Topics==
===[[/Virtualisation]]===
===[[/LDAP]]===
===[[/bind]]===
===[[/ssl]]===
===[[/clamav]]===
===[[/text processing]]

Main Page/IT/clamav

2014-07-02T09:32:57Z

Clewis: /* Create a custom signature */

=Notes on ClamAV=
==Create a custom signature==
This is useful for scanning mail for zip files that contain .EXE and .SCR files which will almost ceratinly be malicious.

Create a file in /var/lib/clamav called zipexe.zmd:
<pre>
ZippedExe:0:.*\.exe:*:*:*:*:*:*
ZippedExe:0:.*\.scr:*:*:*:*:*:*
</pre>

and restart clamd.

Main Page/IT/clamav

2014-07-02T09:32:38Z

Clewis: Created page with "=Notes on ClamAV= ==Create a custom signature== This is useful for scanning mail for zip files that contain .EXE and .SCR files which will almost ceratinly be malicious. Crea..."

Main Page/IT

2014-07-02T09:26:45Z

Clewis:

=IT=
==Topics==
===[[/Virtualisation]]===
===[[/LDAP]]===
===[[/bind]]===
===[[/ssl]]===
===[[/clamav]]===

Main Page/IT

2014-07-02T09:26:32Z

Clewis:

=IT=
==Topics==
===[[/Virtualisation]]===
===[[/LDAP]]===
===[[/bind]]===
===[[/ssl]]===
===[[/clamav]]==

Main Page/IT/ssl

2014-03-14T10:32:19Z

Clewis:

=SSL=

==View expiry date of certificate==
<pre>openssl x509 -noout -in <filename> -dates</pre>

Main Page/IT/ssl

2014-03-14T10:32:11Z

Clewis: Created page with "=SSL= ==View expiry date of certificate== <pre>openssl x509 -noout -in <filename> -dates</pre<"

=SSL=

==View expiry date of certificate==
<pre>openssl x509 -noout -in <filename> -dates</pre<

Main Page/IT

2014-03-14T10:31:16Z

Clewis: /* Topics */

=IT=
==Topics==
===[[/Virtualisation]]===
===[[/LDAP]]===
===[[/bind]]===
===[[/ssl]]===

Main Page/IT/Virtualisation

2014-03-10T13:44:30Z

Clewis: /* Compress a qcow2 disk image */

=KVM=
Stuff relevant to qemu-kvm
==Create a qcow2 disk image==
This will be provisoned "thinly" so the file will grow as data is written into it.

<pre>qemu-img create -f qcow2 mydisk.qcow2 8G</pre>

==Compress a qcow2 disk image==
<pre>qemu-img convert -c -f qcow2 -O qcow2 -o cluster_size=2M myold_disk.qcow2 mynew_disk.qcow2</pre>

==Install an existing VM from the command line using virt-install==
This will spin up a new VM given an existing disk image. One virtio disk and one virtio NIC on br100.

<pre>virt-install --name <<Your machine name>> --hvm --ram 512 --disk path=<<path to your existing OS image>>,format=qcow2,bus=virtio --import --network bridge=br100,model=virtio --vnc --os-type=linux --os-variant=debianwheezy</pre>

==Start a fresh install of VM from command line==

Creates a VM called windows8-test and starts an install of windows from an iso image.

<pre>virt-install --name windows8-test --hvm --ram 1024 --disk path=./windows8-test-disk1.qcow2,format=qcow2,bus=virtio --network bridge=br100,model=virtio --vnc --os-type=windows --cdrom /mnt/vmimages/iso/Windows\ 8\ Retail$MSDN$Pro\ x64\ bit.ISO</pre>

Main Page/IT/Virtualisation

2014-03-10T13:44:20Z

Clewis: /* Create a qcow2 disk image */

=KVM=
Stuff relevant to qemu-kvm
==Create a qcow2 disk image==
This will be provisoned "thinly" so the file will grow as data is written into it.

<pre>qemu-img create -f qcow2 mydisk.qcow2 8G</pre>

==Compress a qcow2 disk image==
<pre>kvm-img convert -c -f qcow2 -O qcow2 -o cluster_size=2M myold_disk.qcow2 mynew_disk.qcow2</pre>

==Install an existing VM from the command line using virt-install==
This will spin up a new VM given an existing disk image. One virtio disk and one virtio NIC on br100.

<pre>virt-install --name <<Your machine name>> --hvm --ram 512 --disk path=<<path to your existing OS image>>,format=qcow2,bus=virtio --import --network bridge=br100,model=virtio --vnc --os-type=linux --os-variant=debianwheezy</pre>

==Start a fresh install of VM from command line==

Creates a VM called windows8-test and starts an install of windows from an iso image.

<pre>virt-install --name windows8-test --hvm --ram 1024 --disk path=./windows8-test-disk1.qcow2,format=qcow2,bus=virtio --network bridge=br100,model=virtio --vnc --os-type=windows --cdrom /mnt/vmimages/iso/Windows\ 8\ Retail$MSDN$Pro\ x64\ bit.ISO</pre>

Main Page/IT/Virtualisation

2014-02-27T10:03:07Z

Clewis: /* Install a VM from the command line using virt-install */

=KVM=
Stuff relevant to qemu-kvm
==Create a qcow2 disk image==
This will be provisoned "thinly" so the file will grow as data is written into it.

<pre>kvm-img create -f qcow2 mydisk.qcow2 8G</pre>

==Compress a qcow2 disk image==
<pre>kvm-img convert -c -f qcow2 -O qcow2 -o cluster_size=2M myold_disk.qcow2 mynew_disk.qcow2</pre>

==Install an existing VM from the command line using virt-install==
This will spin up a new VM given an existing disk image. One virtio disk and one virtio NIC on br100.

<pre>virt-install --name <<Your machine name>> --hvm --ram 512 --disk path=<<path to your existing OS image>>,format=qcow2,bus=virtio --import --network bridge=br100,model=virtio --vnc --os-type=linux --os-variant=debianwheezy</pre>

==Start a fresh install of VM from command line==

Creates a VM called windows8-test and starts an install of windows from an iso image.

<pre>virt-install --name windows8-test --hvm --ram 1024 --disk path=./windows8-test-disk1.qcow2,format=qcow2,bus=virtio --network bridge=br100,model=virtio --vnc --os-type=windows --cdrom /mnt/vmimages/iso/Windows\ 8\ Retail$MSDN$Pro\ x64\ bit.ISO</pre>

Main Page/IT/Virtualisation

2014-02-27T09:59:35Z

Clewis: /* Install a VM from the command line using virt-install */