1. Parameter des Composers bestimmen

Zuerst müssen die Parameter für den Composer bestimmt werden, die erszeugte Datei enthält dann alle benötigten Informationen um Punkte und Bilder, welche unter der gleichen Kamerakonfiguration erstellt wurden, auf ein Koordinatensystem abbzubilden.

Struktur:

$ bb_composer left_img right_img camera_params out_composer_params

konkretes Bsp.:

$ bb_composer Input/Cam_0_20160715130847_631282.jpg Input/Cam_1_20160715130847_631282.jpg Input/camera_params.npz composer_params.npz

(camera_params.npz für die PointGrey Flea3 Camera mit dem Objektiv RICOH FL-CC1214A-2M kann hier runtergeladen werden: https://github.com/gitmirgut/bb_composer/blob/gh-pages/camera_params.npz)

2. Bestimmung der Koordinaten bzgl. des Gesamtbildes

In [1]:
%matplotlib inline
In [2]:
import matplotlib.pyplot as plt
import numpy as np
import composer.core
import cv2
from pipeline import Pipeline
from pipeline.objects import CamParameter, Image, Filename, Positions, HivePositions, CandidateOverlay
from pipeline.pipeline import get_auto_config

Using Theano backend.

Initialisieren der pipeline, wobei als Input neben dem Filename auch CamParameter aufgeührt sein muss.

In [3]:
pipeline = Pipeline([Filename, CamParameter],  # inputs
                    [Image, Positions, HivePositions],  # outputs
                    **get_auto_config())

Laden der Bilder und Composer Parameter:

(Es werden hier zwei Bilder geladen um das Ergebnis später innerhalb eines Panoramas zu überprüfen)

In [4]:
img_l = 'Input/Cam_0_20160715130847_631282.jpg'
img_r = 'Input/Cam_1_20160715130847_631282.jpg'
img_left_org = cv2.imread(img_l)
img_right_org = cv2.imread(img_r)
composer_params = 'composer_params.npz'

Pipline auf das linke, sowie rechte Bild anwenden:

In [5]:
res_l=pipeline([img_l, composer_params])
res_r=pipeline([img_r, composer_params])

Die ursprünglichen Koordinaten bzgl. des rechten Bildes:

In [6]:
res_r[Positions]
Out[6]:
array([[ 1314.90612984,    76.63876724],
       [ 3149.95760727,    73.24475765],
       [ 2374.43832111,   115.27530098],
       [  399.36408043,   150.22371292],
       [ 3424.9200058 ,   163.96108627],
       [ 3149.19769096,   175.08439827],
       [ 2887.40344238,   226.62768173],
       [ 1637.05319023,   248.99994946],
       [ 3162.37470245,   262.92123795],
       [ 1362.15179825,   286.84087849],
       [ 3024.78186607,   312.85179901],
       [ 3787.16011143,   337.72342873],
       [ 1926.28509712,   347.80463982],
       [ 2174.95417595,   374.4863615 ],
       [ 2636.72420692,   389.01831627],
       [ 3349.78386307,   387.68462563],
       [ 3550.8399868 ,   403.48672676],
       [ 1436.85214806,   450.60056114],
       [ 3137.40076256,   486.50326538],
       [ 1874.37572861,   510.49973011],
       [ 3899.75019836,   513.28697586],
       [ 3474.30299568,   537.3659668 ],
       [ 1587.22592163,   549.40079689],
       [ 2099.89573288,   548.94384766],
       [ 1750.28432465,   602.90127945],
       [ 2675.01885796,   601.36205864],
       [ 3186.36732769,   600.05276299],
       [ 1314.29086876,   638.46556854],
       [ 3412.10490799,   637.71415901],
       [  924.28198528,   701.8197937 ],
       [ 1625.35797501,   712.31527901],
       [ 2924.46568298,   712.43743896],
       [ 2126.40532684,   726.412117  ],
       [ 2501.15870285,   726.80298615],
       [ 2774.05100346,   723.99616337],
       [ 3311.61377525,   725.09686852],
       [ 2674.37518787,   750.05347824],
       [ 3101.00816154,   763.17354393],
       [ 1499.81505013,   774.71900272],
       [ 1863.30445671,   774.24073792],
       [  726.87699699,   799.60485649],
       [ 3301.02353859,   815.12943649],
       [ 2249.85581017,   822.6572113 ],
       [ 3024.65451241,   850.30252075],
       [ 3174.83640862,   874.80603409],
       [ 1050.75889206,   888.39504433],
       [ 2524.82856941,   887.95121193],
       [ 2249.55862808,   937.70443916],
       [ 2700.19328117,   936.65529346],
       [ 1426.08927917,   962.44711304],
       [ 3752.36988449,  1001.1927948 ],
       [  987.04280567,  1011.06690979],
       [ 1549.23150826,  1013.99334526],
       [ 2824.81952286,  1012.1375227 ],
       [  686.93064404,  1023.96277809],
       [ 1238.65293121,  1025.23839188],
       [ 2037.60410309,  1024.80564308],
       [ 3412.70635796,  1036.81061554],
       [ 1699.63383484,  1074.7365818 ],
       [ 2238.1090107 ,  1073.64257812],
       [ 3051.86746979,  1088.4199295 ],
       [ 2075.65089989,  1123.01572609],
       [  861.26329613,  1135.76437187],
       [ 2512.77393913,  1148.62658596],
       [ 3337.79029465,  1225.07797813],
       [ 1125.37723541,  1248.01592636],
       [ 2836.45738411,  1251.71354675],
       [ 2524.88169289,  1273.28625202],
       [ 2000.44226074,  1289.73546219],
       [ 2412.11134434,  1287.31707382],
       [ 2150.03027534,  1300.06996536],
       [ 3437.34004593,  1310.6743145 ],
       [ 1337.58149529,  1351.43034935],
       [ 1811.87801361,  1350.23125458],
       [ 1012.2247715 ,  1386.37258053],
       [ 1712.03141499,  1387.65632248],
       [ 2787.51470947,  1426.60353661],
       [  899.59178066,  1437.99214935],
       [ 3037.09920406,  1437.39331245],
       [ 2024.76951408,  1449.21161938],
       [ 3386.84980679,  1475.55461502],
       [ 1051.48391151,  1498.04763985],
       [ 2387.49030876,  1561.97716618],
       [ 2774.66332054,  1575.46929169],
       [ 1925.84738922,  1588.56451797],
       [ 2137.03003693,  1587.15049076],
       [ 2890.09388924,  1588.25680161],
       [  849.58203602,  1625.07023621],
       [ 1762.51414871,  1639.19765663],
       [ 2637.67241478,  1637.98464203],
       [ 1526.849123  ,  1649.86868477],
       [ 2949.46667099,  1650.66033745],
       [ 3474.04802322,  1690.37146378],
       [ 3660.83507061,  1690.41954041],
       [ 2437.87853432,  1698.75360775],
       [ 1111.79604435,  1737.38418198],
       [ 2264.65703773,  1738.04390526],
       [ 1324.88680267,  1775.30620003],
       [ 2038.59000206,  1789.08793831],
       [  987.36121178,  1811.75217152],
       [ 2962.63733673,  1810.43515873],
       [  564.55062103,  1823.60008907],
       [ 2237.49378586,  1889.26684189],
       [ 1925.05178642,  1901.45536613],
       [ 2024.95047188,  1899.9182663 ],
       [  424.23943424,  1911.93375397],
       [ 2487.40019798,  1936.90770054],
       [ 1512.23028564,  1962.13855743],
       [ 3249.79562569,  1977.47785378],
       [  877.77863121,  1988.31302071],
       [ 2788.13999367,  1987.76253128],
       [ 1949.97035217,  2037.81852341],
       [ 2312.77547073,  2038.63476562],
       [ 1762.81475258,  2074.16671944],
       [ 1176.07691193,  2099.10736465],
       [ 2249.63216496,  2125.03850746],
       [ 2537.12499046,  2150.01813507],
       [ 2849.92701721,  2173.94568157],
       [ 1688.36227798,  2251.53005219],
       [ 2263.03561783,  2250.61807251],
       [ 1863.84757614,  2263.18497086],
       [ 2500.83862495,  2276.64255333],
       [ 1987.58217812,  2288.17746544],
       [ 2663.02059364,  2288.24260902],
       [  725.68544006,  2299.51431656],
       [ 1262.05943584,  2312.01535988],
       [ 2312.20084572,  2335.56905842],
       [ 3488.11463928,  2351.39378357],
       [ 3125.84599304,  2363.82324409],
       [  849.76338387,  2373.90246773],
       [ 2850.08943558,  2376.23662376],
       [ 2062.41114426,  2411.25922394],
       [ 2586.30285454,  2412.01777744],
       [ 1626.65639496,  2424.50744343],
       [ 3474.2086134 ,  2437.04263878],
       [ 1552.97513199,  2464.02933311],
       [ 2236.72036362,  2462.24122429],
       [ 2925.25791359,  2462.06542301],
       [ 1751.854599  ,  2477.18824005],
       [ 1424.72345829,  2490.32731628],
       [ 1826.93796349,  2487.76054573],
       [ 2376.11066055,  2486.89495659],
       [ 2014.71234512,  2501.14653397],
       [ 2174.47378731,  2538.08608246],
       [ 2476.61540604,  2562.91799355],
       [ 1687.7670517 ,  2575.4912262 ],
       [ 2026.73410034,  2588.9283905 ],
       [ 3763.35673904,  2590.18698502],
       [  826.24066734,  2610.93418407],
       [ 2588.11338615,  2626.68443871],
       [ 1836.97556305,  2638.14910889],
       [ 1761.31797314,  2651.5743084 ],
       [ 2949.58243847,  2663.71221924],
       [ 3275.6786232 ,  2677.82727814],
       [ 1988.35289192,  2699.37740517],
       [ 3437.03057957,  2700.24218559],
       [  912.40382004,  2724.28432846],
       [  750.0021534 ,  2738.91057014],
       [ 2363.28510475,  2736.96746159],
       [ 2138.04196739,  2762.29763412],
       [ 1564.61112595,  2800.75876808],
       [ 2638.09970474,  2824.82233238],
       [ 2424.33958054,  2837.58812523],
       [ 2237.93673897,  2849.27987099],
       [ 1537.5674839 ,  2874.08244514],
       [ 2064.08824921,  2887.48176575],
       [ 1839.57379532,  2900.60369492],
       [ 1951.6984024 ,  2902.28741455],
       [ 1174.48012924,  2925.29527664],
       [ 1314.04344749,  2925.14621925]])
In [8]:
res_r[HivePositions]
Out[8]:
array([[ 1361.78511797,  5496.20516797],
       [ 3170.78305523,  5506.49496076],
       [ 2412.59713481,  5462.22370878],
       [  436.444563  ,  5417.16156451],
       [ 3439.52711997,  5418.01677659],
       [ 3171.31600987,  5406.15466187],
       [ 2916.50146422,  5354.27907809],
       [ 1685.09681397,  5325.89355288],
       [ 3185.26682236,  5319.62746906],
       [ 1411.18150045,  5286.51022038],
       [ 3051.70788687,  5269.78052642],
       [ 3793.60002671,  5248.22903861],
       [ 1973.18315226,  5229.22798154],
       [ 2219.13784844,  5204.22825852],
       [ 2673.1538148 ,  5192.52452869],
       [ 3369.41563484,  5197.44865895],
       [ 3565.16238353,  5182.7219447 ],
       [ 1487.49024254,  5123.41360009],
       [ 3163.78893514,  5098.9453453 ],
       [ 1923.49352502,  5066.97675926],
       [ 3905.37956585,  5075.96724595],
       [ 3492.62584588,  5050.49274949],
       [ 1638.52847094,  5025.92988083],
       [ 2146.99484563,  5030.41236976],
       [ 1801.31167204,  4973.90974   ],
       [ 2713.21619522,  4982.53652089],
       [ 3213.03856294,  4987.09990815],
       [ 1366.57911902,  4934.42682596],
       [ 3433.50765306,  4951.20031376],
       [  974.37938806,  4866.75830384],
       [ 1678.11521353,  4863.56259616],
       [ 2958.76710793,  4874.29766113],
       [ 2175.10237463,  4854.11934415],
       [ 2544.13737219,  4856.87312431],
       [ 2811.73211268,  4861.76378524],
       [ 3336.84683229,  4864.33901151],
       [ 2714.39185726,  4835.17935573],
       [ 3131.92648428,  4825.32375268],
       [ 1553.47356764,  4799.89828326],
       [ 1915.24719912,  4804.04640296],
       [  774.89806836,  4765.85832657],
       [ 3327.78897743,  4775.29422315],
       [ 2297.9583521 ,  4759.47051253],
       [ 3058.50866192,  4738.53662201],
       [ 3205.52479018,  4715.38739711],
       [ 1103.59978322,  4680.59775521],
       [ 2569.30096482,  4697.06426761],
       [ 2298.95125136,  4644.96121737],
       [ 2742.01493529,  4650.26001522],
       [ 1481.42972727,  4610.98102773],
       [ 3770.03153256,  4594.24988183],
       [ 1040.1254919 ,  4556.19614388],
       [ 1604.99898591,  4560.84813368],
       [ 2865.12402941,  4576.43087736],
       [  735.65532448,  4538.67249278],
       [ 1293.91527402,  4545.46919202],
       [ 2090.5173884 ,  4555.8695521 ],
       [ 3439.90851645,  4556.74974477],
       [ 1755.53208734,  4501.89801874],
       [ 2289.17464386,  4509.38770905],
       [ 3088.36216267,  4502.71735338],
       [ 2129.19909719,  4458.24947777],
       [  913.53414765,  4428.41914422],
       [ 2560.59322445,  4437.65404453],
       [ 3369.69531614,  4369.54331134],
       [ 1181.4612401 ,  4319.38488989],
       [ 2879.70467345,  4338.41631973],
       [ 2574.02462328,  4313.54797315],
       [ 2056.45509642,  4290.56855653],
       [ 2463.17928041,  4298.22815893],
       [ 2204.65911434,  4282.21139565],
       [ 3468.18808372,  4285.40747662],
       [ 1395.79186787,  4218.60519062],
       [ 1869.81135043,  4227.2627832 ],
       [ 1067.99663679,  4177.61254063],
       [ 1770.72838655,  4188.16848486],
       [ 2834.06286639,  4163.63111874],
       [  954.12281209,  4123.19961782],
       [ 3078.90624524,  4155.60053696],
       [ 2082.19791732,  4131.08932545],
       [ 3421.55463544,  4121.05609741],
       [ 1108.36042778,  4065.28883669],
       [ 2442.16843914,  4023.2418377 ],
       [ 2823.47914152,  4014.81562855],
       [ 1985.38596601,  3989.58189673],
       [ 2194.87746824,  3994.33497113],
       [ 2936.98831396,  4003.42946108],
       [  904.19471793,  3932.21122053],
       [ 1823.26869283,  3935.8383247 ],
       [ 2689.63449333,  3950.51307552],
       [ 1587.84366771,  3920.79605617],
       [ 2996.14865834,  3941.76250715],
       [ 3510.33068925,  3907.83101391],
       [ 3692.69210349,  3909.57171205],
       [ 2493.53656669,  3886.80169473],
       [ 1170.66070468,  3823.7436277 ],
       [ 2322.86135244,  3844.69758987],
       [ 1385.89228271,  3789.79862268],
       [ 2099.3570033 ,  3789.56313367],
       [ 1044.9618245 ,  3745.31500641],
       [ 3011.45594354,  3781.99087649],
       [  613.84775919,  3723.00874607],
       [ 2297.6766266 ,  3691.99247987],
       [ 1987.56546232,  3674.07217695],
       [ 2086.91781342,  3677.48497444],
       [  469.95966856,  3628.85049945],
       [ 2545.38103439,  3648.11361532],
       [ 1575.4302934 ,  3604.18498093],
       [ 3295.79526332,  3618.0849932 ],
       [  934.25569416,  3562.48206006],
       [ 2842.54716533,  3601.53161608],
       [ 2013.65782952,  3536.55442474],
       [ 2373.94224585,  3542.55171363],
       [ 1827.25492497,  3495.82921364],
       [ 1237.45624765,  3456.8065263 ],
       [ 2312.35661326,  3454.01701643],
       [ 2597.24272502,  3433.91241357],
       [ 2906.1239841 ,  3414.85626025],
       [ 1754.14281671,  3313.71542724],
       [ 2327.07639947,  3326.99732406],
       [ 1929.8014492 ,  3305.85294757],
       [ 2563.01602675,  3305.1143984 ],
       [ 2053.50306155,  3283.14026827],
       [ 2723.47445472,  3296.25435468],
       [  779.78338493,  3238.99995905],
       [ 1325.42137623,  3241.238872  ],
       [ 2376.84458241,  3241.71834696],
       [ 3536.01937651,  3245.01110156],
       [ 3180.65683577,  3227.30712681],
       [  906.7440991 ,  3166.07439345],
       [ 2909.34965421,  3210.33777649],
       [ 2129.28634732,  3159.42694748],
       [ 2649.35986196,  3169.36665394],
       [ 1693.49005785,  3135.53622699],
       [ 3523.99536288,  3158.38148504],
       [ 1619.68448323,  3093.16577151],
       [ 2303.32666003,  3111.26881199],
       [ 2984.79002564,  3124.61959006],
       [ 1819.49916322,  3084.77090818],
       [ 1490.6263404 ,  3062.76780065],
       [ 1894.79796557,  3075.80924835],
       [ 2442.07171734,  3089.0851966 ],
       [ 2082.59911812,  3066.61097751],
       [ 2242.22653083,  3032.53700824],
       [ 2542.72589511,  3013.69879887],
       [ 1755.93419506,  2982.41184304],
       [ 2095.44465984,  2977.14236367],
       [ 3810.16747604,  3007.64196008],
       [  883.17301382,  2920.73991176],
       [ 2654.08081269,  2950.96623308],
       [ 1906.09395263,  2922.00657042],
       [ 1830.33719447,  2906.25998076],
       [ 3011.99696632,  2920.19228338],
       [ 3333.48083887,  2911.49323465],
       [ 2058.14677991,  2863.03976473],
       [ 3492.46605063,  2891.34660722],
       [  971.44404998,  2806.20861742],
       [  805.30672157,  2785.69105238],
       [ 2432.36759081,  2833.47424974],
       [ 2208.28648749,  2802.20480281],
       [ 1633.51794178,  2747.29075554],
       [ 2706.36836233,  2749.62132708],
       [ 2494.31528904,  2731.7974619 ],
       [ 2308.86615034,  2715.37586942],
       [ 1606.68289087,  2670.80727096],
       [ 2135.6803092 ,  2671.7147164 ],
       [ 1910.85267051,  2652.14576101],
       [ 2023.31959304,  2653.48701945],
       [ 1239.08554173,  2606.44476336],
       [ 1380.81370575,  2611.096654  ]])

3. bildliche Überprüfung

Zusammenführung der Punkte aus dem linken sowie rechten Bild:

In [9]:
pts_l = res_l[HivePositions]
pts_r = res_r[HivePositions]
pts = np.vstack((pts_l, pts_r))
pts[:,[0, 1]] = pts[:,[1, 0]] #  swap y, x --> x, y
In [10]:
c = composer.core.Composer()
c.load_arguments(composer_params)
res = c.compose_and_mark(img_left_org,img_right_org, np.array([pts]))
In [11]:
plt.figure(figsize=(16, 16))
plt.imshow(res, cmap=plt.cm.gray)
plt.grid()
plt.show()

cv2.imwrite('result.jpg', res)