Fagidiot.dk: 2015

søndag den 13. december 2015

JHead Rename images based on EXIF info

I just discovered JHead, a tool for manipulating JPEG images based on EXIT information. I don't know why I didn't find this tool before. This is extremely useful and convenient. I have tried a couple of tools on Windows and Mac but nothing is as convient at this one.

Install using homebrew:

brew install libjpeg
brew install jhead

libjpeg includes jpgtran, which is prerequisite for jhead but did not install as part of the jhead install.

I name my files using a date timestamp:

jhead -autorot -nf%Y-%m-%d-%Hh%Mm%S *.jpg

References:
http://www.phpied.com/installing-jpegtran-mac-unix-linux/
http://www.sentex.ca/~mwandel/jhead/
http://moinne.com/blog/ronald/bash/rename-your-photos-from-the-comandline-using-jhead

tirsdag den 8. december 2015

Release it

I am reading Release It by Nygard and these are some of the things to remember:

Use limits on queries. E.g. LIMIT 1 on Postgres or MySQL
Watch out for point-point connections
Set Socket.setSoTimeout() to protect against servers under heavy load
Use circuit breakers to degrade functionality gracefully under load
Return different errors for resource not available and application failures
Fail fast, check availability of all required resources (ram, db, integration points, input)
Purge data (database, caches, logs)
Steady state
Bulkheads
Cooperative demand control, handshaking between client and server to allow clients to back off. Problematic with http as the setup and teardown of connections is often the most expensive. Alternatively consider health checks either from client or from load balancer. Build handshaking into lowlevel protocols.

SublimeText 2 Shortcuts

I am new to SublimeText 2 and so I need a little reminding of various shortcuts. Multiple cursors is one of the things I loved the most in Emacs. So I find this page very useful.

Simple overview of selection.

torsdag den 26. november 2015

Git Rebase Commands

The goal of this post is to make it easier for me to reference the individual steps of rebasing a Git repository.

I usually push the branch to origin as soon as I have made the first commit for backup purposes. Then I do an interactive rebase when the feature is complete to clean up the branch.

First update develop:

git checkout develop
git pull -u
git checkout feature/branch

Perform the interactive rebase:

git rebase -i develop

Push the feature branch to origin using force. Don't do this if somebody else is working on the same branch.

git push --force

If you don't want to do the rebase interactively, then just rebase:
git rebase develop

git rebase develop

If your current branch contain commits already merged into develop. Or is the continuation of a previous branch merged with develop after the current branch has been created. Then you must apply a little more magic.
Get the hash of the commit immediately before the commits you want to keep. The run the following command with #hash replaced by the hash of the commit.

git rebase --onto master #hash

The changes must be force-pushed.

See more here: https://www.atlassian.com/git/tutorials/merging-vs-rebasing/workflow-walkthrough.

mandag den 16. november 2015

How to list all databases with a given prefix in PostgeSQL

I recently had to delete a large number of databases in a PostgreSql database.

I found the code below very useful. It will list a sequence of SQL DROP DATABASE statements, one for each of the databases with a name starting with "database_name_starting_with_this".

The list of statements can be copied to psql a executed by pasting.

I want to warn you this can be dangerous, so go through the list of statements one by one, before you execute them.

SELECT 'DROP DATABASE "' || datname || '";' FROM pg_database
WHERE datname LIKE 'database_name_starting_with_this%';

fredag den 18. september 2015

How to remove deleted branch names from Git autocomplete

This is actually just a kind of short-cut to this SO. But if you want to skip the reading then here is the one-liner which does the cleaning for you.

git fetch --prune --all

Also to prune all local branches which have already been merged:

git branch -d $(git branch --merged)
git remote prune origin
git branch -d $(git branch | sed '$d')

onsdag den 16. september 2015

How to generate a cryptographically secure random string in JavaScript

When working with security and especially cryptography one often needs to generate a sequence of bytes which should be uniformly random. This is the case when creating a nonce for OAuth, and many other applications.

This problem is often mistaken for the problem of generating a random string. However, a random sequence of bytes is not a string. But can be represented as a string. And often this is exactly what you want, hence the name of this post.

If I use Google to search for "js random string", then I will get many suggestions on how to generate a random string in JavaScript. However, they are either not cryptographically secure or they rely on Node.js. I will ignore the Node.js versions as they seems to fine.

So what is wrong with the many insecure solutions:

They use Math.random() which is not cryptographically secure.
They tend to use a reduced alphabet of only alphanumeric characters

One example of the insecure random is this from this SO:

function makeid()
{
    var text = "";
    var possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";

    for( var i=0; i < 5; i++ )
        text += possible.charAt(Math.floor(Math.random() * possible.length));

    return text;
}

A better approach is to use the window.crypto.getRandomValues method from the WebCrypto API, which states that the Browser is supposed to use a strong strong (pseudo) random number generator. And then encode the result as Base64. Notice that this method will generate a random sequence of numberOfBytes bytes and then base64 encode these. This will generally result in a string of more than numberOfBytes characters.

function randomString (numberOfBytes) {
    var array = new Uint8Array(length);
    window.crypto.getRandomValues(array);
    return btoa(String.fromCharCode.apply(null, array));
}

This approach is not foolproof either. As mentioned in this SO comment you might not want to trust the client to generate randomness. However, there are situations where you want to, so the call is up to you.

fredag den 6. marts 2015

Sending a WM_COPYDATA from C++ to Python

The following code shows how to send a WM_DATACOPY message from C++ and how to recede it in Python.

The code is based on the code in these posts: http://stackoverflow.com/questions/2451103/use-wm-copydata-to-send-data-between-processes, and http://stackoverflow.com/questions/5249903/receiving-wm-copydata-in-python.

Sending in C++:

LPTSTR lpszString = L"Not ready";
COPYDATASTRUCT cds;
cds.dwData = 1;
size_t count_t = sizeof(TCHAR) * (_tcslen(lpszString) + 1);
cds.cbData = static_cast<DWORD>(count_t);
cds.lpData = lpszString;
SendMessage(hwnd, WM_COPYDATA, (WPARAM)hwnd, (LPARAM)(LPVOID)&cds);

Recieving in Python:

import win32api
import win32process

import struct
import win32con
import win32gui
import array
import win32ui
from ctypes import *


class ACOPYDATASTRUCT(Structure):
    _fields_ = [
        ('dwData', c_ulong),
        ('cbData', c_ulong),
        ('lpData', c_void_p)
    ]
PCOPYDATASTRUCT = POINTER(ACOPYDATASTRUCT)


class Listener:

    def __init__(self):
        message_map = {
            win32con.WM_COPYDATA: self.OnCopyData
        }
        wc = win32gui.WNDCLASS()
        wc.lpfnWndProc = message_map
        wc.lpszClassName = 'MyWindowClass'
        hinst = wc.hInstance = win32api.GetModuleHandle(None)
        classAtom = win32gui.RegisterClass(wc)
        self.hwnd = win32gui.CreateWindow (
            classAtom,
            "win32gui test",
            0,
            0, 
            0,
            win32con.CW_USEDEFAULT, 
            win32con.CW_USEDEFAULT,
            0, 
            0,
            hinst, 
            None
        )
        print self.hwnd

    def OnCopyData(self, hwnd, msg, wparam, lparam):
        print hwnd
        print msg
        print wparam
        print lparam
        pCDS = cast(lparam, PCOPYDATASTRUCT)
        print pCDS.contents.dwData
        print pCDS.contents.cbData
        status = wstring_at(pCDS.contents.lpData)
        print status
        win32gui.PostQuitMessage(0)
        return 1

l = Listener()

win32gui.PumpMessages()

As an additional bonus one can send the hwnd of the receiving python program to the C++ program using a WM_COPYDATA message:

class COPYDATATYPE(Structure):
    _fields_ = [("cmd",   c_ulong),
                ("hwnd",  c_ulong)]

class COPYDATASTRUCT(Structure):
    _fields_ = [("dwData", c_ulong),
                ("cbData", c_ulong),
                ("lpData", POINTER(COPYDATATYPE))]

cpyData = COPYDATATYPE(1, l.hwnd)
cds = COPYDATASTRUCT(c_ulong(8888),
                     c_ulong(sizeof(cpyData)),
                     pointer(cpyData))

# try to send a message
win32api.SendMessage(target_hwnd,
                     win32con.WM_COPYDATA,
                     0,
                     addressof(cds))

See also my previous post here: http://blog.fagidiot.dk/2015/03/send-wmcopydata-from-python-partial.html

torsdag den 5. marts 2015

Formatting code on this blog

I use http://hilite.me if you have alternative suggestions then don't hesitate to leave a comment.

Send WM_COPYDATA from Python - Partial Solution

I wanted to send a WM_COPYDATA message from a Python application to a Windows application.

Here is the code both Python and C++ which I got work with the only caveat that the string in szData is not received correctly. This example is a very tiny modification of the code posted here: https://groups.google.com/forum/#!topic/comp.lang.python/z3QS7Ic80mU

Python code:

import struct
import win32con
import win32gui
import array

class COPYDATATYPE(Structure):
    _fields_ = [("nNum",   c_ulong),
                ("szData", c_char_p)]

class COPYDATASTRUCT(Structure):
    _fields_ = [("dwData", c_ulong),
                ("cbData", c_ulong),
                ("lpData", POINTER(COPYDATATYPE))]

cpyData = COPYDATATYPE(17, '123456789')
cds = COPYDATASTRUCT(c_ulong(1),
                     c_ulong(sizeof(cpyData)),
                     pointer(cpyData))

# try to send a message
win32api.SendMessage(hwnd,
                     win32con.WM_COPYDATA,
                     0,
                     addressof(cds))

C++ code:

 case WM_COPYDATA:
    pCDS = (COPYDATASTRUCT*)lParam;

    if (pCDS->dwData == 1)
    {
      struct COPYDATATYPE
      {
        long nNum;
        char * szData;
      };

      int count = pCDS->cbData;
      PVOID data = pCDS->lpData;
      if (data != NULL)
      {
        COPYDATATYPE *cp = (COPYDATATYPE *)data;
        int i = cp->nNum;
        // The following line does not work.
        const char* message = cp->szData;
      }
    }
    break;

I also found use in reading this: http://www.qsl.net/dl4yhf/yhf_comm/yhf_comm_info.htm#find_window_handle

Get the list of hwnds for a given pid

This Python code shows how to get the list of hwnds from a given pid on Windows.

We use win32gui to enumerate all windows, get their pid to check and then append the hwnd to the list.

def get_hwnds_for_pid (pid):
  def callback (hwnd, hwnds):
    _, found_pid = win32process.GetWindowThreadProcessId (hwnd)
    if found_pid == pid:
        hwnds.append (hwnd)
    return True
    
  hwnds = []
  win32gui.EnumWindows (callback, hwnds)
  return hwnds

hwnds = get_hwnds_for_pid(3196)
print hwnds