Generating fake data can be a common need when developing applications or loading test data into a database. jFairy is a great fake data generator library built in Java that is very easy to use. jFairy allows you to build data sets containing diverse types of data including names, addresses, telephone numbers, dates, large integers,
A fat jar or uber jar is a jar that contains the classes of your current project as well as all of the classes on which it depends. For example, if your application requires Joda-Time, your jar file will contain all the classes of your current project, as well as all the classes of Joda-Time.
Often when working with files in S3, you need information about all the items in a particular S3 bucket. Below is an example class that extends the AmazonS3Client class to provide this functionality. For the most part this class has been adapted from the sample in this AWS post. Aside from some additional methods, one
Exporting data from Postgres as a CSV or tab delimited file can be fairly simple using the COPY command. Examples below. Export an entire table as CSV:
|
1 |
psql -c "COPY my_table TO STDOUT WITH CSV" > output.csv |
Export an entire table as CSV with Header:
|
1 |
psql -c "COPY my_table TO STDOUT WITH CSV HEADER" > output.csv |
Export query as CSV:
|
1 |
psql -c "COPY (SELECT first_name, last_name FROM my_table) TO STDOUT WITH CSV" > output.csv |
Export query as data tab delimited text file:
|
1 |
psql -c "COPY (SELECT first_name, last_name FROM my_table) TO STDOUT" > output.txt |
Export data directly to
Currently AWS CLI doesn’t provide support for UNIX wildcards in a command’s “path” argument. However, it is quite easy to replicate this functionality using the –exclude and –include parameters available on several aws s3 commands. The wildcards available for use are: “*” – Matches everything “?” – Matches any single character “[]” – Matches any
When working with Docker in Linux, you may find yourself using sudo before every Docker command. This is because the docker daemon binds to a Unix socket (which by default is owned by the root user) rather than a TCP port. Thus the reason for always needing to use sudo. A good way to avoid
By default, the root user is locked in Ubuntu. To unlock it, simply run the command below, and provide the new password for the root user when prompted.
|
1 2 3 4 |
sudo passwd root [sudo] password for nik: Enter new UNIX password: Retype new UNIX password: |
Now you should be able to log in as the root user using the su command:
|
1 |
$ su - |
If for some reason you want to lock the
Counting the distinct/unique elements of text file is a common task. Below is an example of doing this is AWK, using sample_data_1.txt.
|
1 2 |
cat sample_data_1.txt \ | awk 'BEGIN{FS="\t"} NR>1{names[$2]=1} END{print length(names)}' |
Here is what is happening above: cat sample_data_1.txt – reading the file piping the data to AWK BEGIN{FS=”\t”} – specifying the field separators of the file NR>1 – Only executing the following