Uk gambling industry profits definition. We've got a django-cms site that started out on sqlite. Now we're adding GISshapefiles so we wanted to move to postgres/postgis. Ok, switch over theDATABASES definition in settings.py, make a fixture and load it,right? Ahem, no.
There were a couple of problems, partially with our code, partiallydjango-cms, partially django. Some solutions below might not be needed inevery case, but I'm just going to list everything in the hope that it helpssomeone.
There are too many casino games to count and always more, is the simple Cms Reinout Slot answer. A quick Cms Reinout Slot overview of the kinds of games you can typically expect from a casino includes slots, poker, blackjack, roulette, scratch cards, bingo, keno, craps and baccarat, to name but a few. Migrating django(-cms) from sqlite to postgres¶ Tags: django. We've got a django-cms site that started out on sqlite. Now we're adding GIS shapefiles so we wanted to move to postgres/postgis. Ok, switch over the DATABASES definition in settings.py, make a fixture and load it, right?
Reinout Slot reinout.slot@cms-dsb.com + 319: Hotels & Leisure: Roman Tarlavski roman.tarlavski@cms-dsb.com + 312. Reinout Slot is the key contact at CMS for investment funds work; he has expertise in structuring, fund formation and operation, and fundraising activities. Finnius Group head Rosemarijn Labeur, who has ' in-depth knowledge of AIFMD, MIFID II and UCITS ', assisted Momentum Capital and Hanzevast Capital with a variety of regulatory matters. Reinout Slot Partner. Amsterdam + 319. CMS UK, with offices in London, Bristol, Sheffield, Manchester, Reading, Aberdeen, Edinburgh.
South migration dependencies¶
Lots of our code uses south for databasemigrations. But when creating our database from scratch, 'migrate' wouldcomplain that a certain table of application A didn't exist yet when migratingapplication B. Uh oh, a dependency.
Turns out that's easily solved. In the failing migration step, just add adependency on the missing application's relevant step:
Reinout Slot Cms Login
After that, all migrations run fine.
Small sqldiff limitation¶
We've got django-extensions in our projects. One of thehandy extra management commands itprovides is 'sqldiff' (so manage.pysqldiff or, as we use buildout,bin/djangosqldiff). (The best extension is 'graph_models' which createsa picture of your model structure).
We got some complaints when importing a fixture. In the end it turned out tobe that someone somehow copied an old and wrong fixture instead of a freshone. But we suspected that some older migration had mis-fired. 'sqldiff'will tell you the difference between the current database schema inside theactual database and what it should be according to django.
We got a bunch of missing fields in some custom user model inside django-cms!What! Faulty south migrations? What?
In the end, it turned out to be a false alarm. Django-cms's user modelinherits from django's own user model and adds one field. In the database thisends up as a small table with a pointer to a corresponding django user item'sID and the one single extra field. But sqldiff complains about all the regulardjango user model fields being missing :-)
Note: I don't fault django-extensions very much here. They note it is anexperimental feature and we just happened to hit a corner case. Most of thetime it is a real helper!
IntegrityError: duplicate key value violates unique constraint¶
See for instance a stackoverflow questionand especially Mark van Lent's recent explanation of what can happen.
There are two basic ways, I think, such an IntegrityError can crop up whenloading a fixture:
You're loading them out of order or with gaps in the primary key IDs. Thiscan mess up the automatic ID generation.
You've really already got an item with that ID in the database and thecomplaint is right.
IntegrityError solution 1: natural keys¶
See django ticket #7052. Solved. Solved, that is, whenthe model that gives you a problem supports so-called naturalkeys. Basically, your fixture doesn't point at a user with primary key '54',but at a user with username 'pietje', whatever the ID.
This takes away a too-hard dependency on primary key ID numbers. To get itgoing in your fixture, add the --natural flag:
IntegrityError solution 2: make sure there are no duplicates¶
Reinout Slot Cms Definition
One of our problems was that django's content_type table was alreadypre-filled with several items after the regular 'syncdb'. Quite sensibly. Butit conflicted when loading our fixture. (It might be that the natural keysolution mentioned above would have solved this, but we tried it in adifferent order).
The solution: remove those content types from the table before loading thefixture. The fixture is a full fixture of everything in the database, sothat's OK.
In your manage.pydbshell or bin/djangodbshell, run:
IntegrityError solution 3: split the fixture¶
Django-cms's cms.placeholder items continued to give grief no matterwhat. Perhaps they ought to start to support the natural key solution? Notsure if that would help, but I guess so.
Reinout Slot Cms App
The solution? Grab the whole exported fixture and split that all.json intotwo json files, one with only 'cms.placeholder' items and one withoutthem. (Make sure the last dict-like item in the json should be without aclosing comma!
all_only_placeholders.json:
all_without_placeholders.json:
After that, load the data:
Closing comment¶
In the end, we got it running. I hope someone gets a good tip out of thisentry when faced with a similar problem.
There were a couple of problems, partially with our code, partiallydjango-cms, partially django. Some solutions below might not be needed inevery case, but I'm just going to list everything in the hope that it helpssomeone.
There are too many casino games to count and always more, is the simple Cms Reinout Slot answer. A quick Cms Reinout Slot overview of the kinds of games you can typically expect from a casino includes slots, poker, blackjack, roulette, scratch cards, bingo, keno, craps and baccarat, to name but a few. Migrating django(-cms) from sqlite to postgres¶ Tags: django. We've got a django-cms site that started out on sqlite. Now we're adding GIS shapefiles so we wanted to move to postgres/postgis. Ok, switch over the DATABASES definition in settings.py, make a fixture and load it, right?
Reinout Slot reinout.slot@cms-dsb.com + 319: Hotels & Leisure: Roman Tarlavski roman.tarlavski@cms-dsb.com + 312. Reinout Slot is the key contact at CMS for investment funds work; he has expertise in structuring, fund formation and operation, and fundraising activities. Finnius Group head Rosemarijn Labeur, who has ' in-depth knowledge of AIFMD, MIFID II and UCITS ', assisted Momentum Capital and Hanzevast Capital with a variety of regulatory matters. Reinout Slot Partner. Amsterdam + 319. CMS UK, with offices in London, Bristol, Sheffield, Manchester, Reading, Aberdeen, Edinburgh.
South migration dependencies¶
Lots of our code uses south for databasemigrations. But when creating our database from scratch, 'migrate' wouldcomplain that a certain table of application A didn't exist yet when migratingapplication B. Uh oh, a dependency.
Turns out that's easily solved. In the failing migration step, just add adependency on the missing application's relevant step:
Reinout Slot Cms Login
After that, all migrations run fine.
Small sqldiff limitation¶
We've got django-extensions in our projects. One of thehandy extra management commands itprovides is 'sqldiff' (so manage.pysqldiff or, as we use buildout,bin/djangosqldiff). (The best extension is 'graph_models' which createsa picture of your model structure).
We got some complaints when importing a fixture. In the end it turned out tobe that someone somehow copied an old and wrong fixture instead of a freshone. But we suspected that some older migration had mis-fired. 'sqldiff'will tell you the difference between the current database schema inside theactual database and what it should be according to django.
We got a bunch of missing fields in some custom user model inside django-cms!What! Faulty south migrations? What?
In the end, it turned out to be a false alarm. Django-cms's user modelinherits from django's own user model and adds one field. In the database thisends up as a small table with a pointer to a corresponding django user item'sID and the one single extra field. But sqldiff complains about all the regulardjango user model fields being missing :-)
Note: I don't fault django-extensions very much here. They note it is anexperimental feature and we just happened to hit a corner case. Most of thetime it is a real helper!
IntegrityError: duplicate key value violates unique constraint¶
See for instance a stackoverflow questionand especially Mark van Lent's recent explanation of what can happen.
There are two basic ways, I think, such an IntegrityError can crop up whenloading a fixture:
You're loading them out of order or with gaps in the primary key IDs. Thiscan mess up the automatic ID generation.
You've really already got an item with that ID in the database and thecomplaint is right.
IntegrityError solution 1: natural keys¶
See django ticket #7052. Solved. Solved, that is, whenthe model that gives you a problem supports so-called naturalkeys. Basically, your fixture doesn't point at a user with primary key '54',but at a user with username 'pietje', whatever the ID.
This takes away a too-hard dependency on primary key ID numbers. To get itgoing in your fixture, add the --natural flag:
IntegrityError solution 2: make sure there are no duplicates¶
Reinout Slot Cms Definition
One of our problems was that django's content_type table was alreadypre-filled with several items after the regular 'syncdb'. Quite sensibly. Butit conflicted when loading our fixture. (It might be that the natural keysolution mentioned above would have solved this, but we tried it in adifferent order).
The solution: remove those content types from the table before loading thefixture. The fixture is a full fixture of everything in the database, sothat's OK.
In your manage.pydbshell or bin/djangodbshell, run:
IntegrityError solution 3: split the fixture¶
Django-cms's cms.placeholder items continued to give grief no matterwhat. Perhaps they ought to start to support the natural key solution? Notsure if that would help, but I guess so.
Reinout Slot Cms App
The solution? Grab the whole exported fixture and split that all.json intotwo json files, one with only 'cms.placeholder' items and one withoutthem. (Make sure the last dict-like item in the json should be without aclosing comma!
all_only_placeholders.json:
all_without_placeholders.json:
After that, load the data:
Closing comment¶
In the end, we got it running. I hope someone gets a good tip out of thisentry when faced with a similar problem.
One django advantage I noticed: everything is visible. You can alwaysinspect the source code. 'What exactly is in that one model that's giving megrief?' 'Which migration steps are available?' Etc.
The advantage of open source!
