-
-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Shillelagh only analyzes 64 tags in a superset #467
Comments
Generation was performed by the activity process. var test = [];
for (var l = 0; l < 2; l++) {
var o = {};
for (var i = 0; i < 100; i++) {
o["col_" + i] = i + (100 * l);
}
test.push(o);
}
execution.setVariable("test", JSON.stringify(test)); |
@mibarannikov what version are you running? I just tested with 1.2.26 and it works:
Where the data came from this Flask app: from flask import Flask
app = Flask(__name__)
@app.route("/data", methods=["GET"])
def get_data():
return (
'[{"col_0":0,"col_1":1,"col_2":2,"col_3":3,"col_4":4,"col_5":5,"col_6":6,"col_7":7,"col_8":8,"col_9":9,"col_10":10,"col_11":11,"col_12":12,"col_13":13,"col_14":14,"col_15":15,"col_16":16,"col_17":17,"col_18":18,"col_19":19,"col_20":20,"col_21":21,"col_22":22,"col_23":23,"col_24":24,"col_25":25,"col_26":26,"col_27":27,"col_28":28,"col_29":29,"col_30":30,"col_31":31,"col_32":32,"col_33":33,"col_34":34,"col_35":35,"col_36":36,"col_37":37,"col_38":38,"col_39":39,"col_40":40,"col_41":41,"col_42":42,"col_43":43,"col_44":44,"col_45":45,"col_46":46,"col_47":47,"col_48":48,"col_49":49,"col_50":50,"col_51":51,"col_52":52,"col_53":53,"col_54":54,"col_55":55,"col_56":56,"col_57":57,"col_58":58,"col_59":59,"col_60":60,"col_61":61,"col_62":62,"col_63":63,"col_64":64,"col_65":65,"col_66":66,"col_67":67,"col_68":68,"col_69":69,"col_70":70,"col_71":71,"col_72":72,"col_73":73,"col_74":74,"col_75":75,"col_76":76,"col_77":77,"col_78":78,"col_79":79,"col_80":80,"col_81":81,"col_82":82,"col_83":83,"col_84":84,"col_85":85,"col_86":86,"col_87":87,"col_88":88,"col_89":89,"col_90":90,"col_91":91,"col_92":92,"col_93":93,"col_94":94,"col_95":95,"col_96":96,"col_97":97,"col_98":98,"col_99":99},{"col_0":100,"col_1":101,"col_2":102,"col_3":103,"col_4":104,"col_5":105,"col_6":106,"col_7":107,"col_8":108,"col_9":109,"col_10":110,"col_11":111,"col_12":112,"col_13":113,"col_14":114,"col_15":115,"col_16":116,"col_17":117,"col_18":118,"col_19":119,"col_20":120,"col_21":121,"col_22":122,"col_23":123,"col_24":124,"col_25":125,"col_26":126,"col_27":127,"col_28":128,"col_29":129,"col_30":130,"col_31":131,"col_32":132,"col_33":133,"col_34":134,"col_35":135,"col_36":136,"col_37":137,"col_38":138,"col_39":139,"col_40":140,"col_41":141,"col_42":142,"col_43":143,"col_44":144,"col_45":145,"col_46":146,"col_47":147,"col_48":148,"col_49":149,"col_50":150,"col_51":151,"col_52":152,"col_53":153,"col_54":154,"col_55":155,"col_56":156,"col_57":157,"col_58":158,"col_59":159,"col_60":160,"col_61":161,"col_62":162,"col_63":163,"col_64":164,"col_65":165,"col_66":166,"col_67":167,"col_68":168,"col_69":169,"col_70":170,"col_71":171,"col_72":172,"col_73":173,"col_74":174,"col_75":175,"col_76":176,"col_77":177,"col_78":178,"col_79":179,"col_80":180,"col_81":181,"col_82":182,"col_83":183,"col_84":184,"col_85":185,"col_86":186,"col_87":187,"col_88":188,"col_89":189,"col_90":190,"col_91":191,"col_92":192,"col_93":193,"col_94":194,"col_95":195,"col_96":196,"col_97":197,"col_98":198,"col_99":199}]',
{"Content-Type": "application/json"},
)
if __name__ == "__main__":
app.run(debug=True) |
2024-08-29T20:51:33+03:00 2024-08-29 17:51:33,485:DEBUG:shillelagh.adapters.api.generic_json:{'col_0': 0, 'col_1': 1, 'col_2': 2, 'col_3': 3, 'col_4': 4, 'col_5': 5, 'col_6': 6, 'col_7': 7, 'col_8': 8, 'col_9': 9, 'col_10': 10, 'col_11': 11, 'col_12': 12, 'col_13': 13, 'col_14': 14, 'col_15': 15, 'col_16': 16, 'col_17': 17, 'col_18': 18, 'col_19': 19, 'col_20': 20, 'col_21': 21, 'col_22': 22, 'col_23': 23, 'col_24': 24, 'col_25': 25, 'col_26': 26, 'col_27': 27, 'col_28': 28, 'col_29': 29, 'col_30': 30, 'col_31': 31, 'col_32': 32, 'col_33': 33, 'col_34': 34, 'col_35': 35, 'col_36': 36, 'col_37': 37, 'col_38': 38, 'col_39': 39, 'col_40': 40, 'col_41': 41, 'col_42': 42, 'col_43': 43, 'col_44': 44, 'col_45': 45, 'col_46': 46, 'col_47': 47, 'col_48': 48, 'col_49': 49, 'col_50': 50, 'col_51': 51, 'col_52': 52, 'col_53': 53, 'col_54': 54, 'col_55': 55, 'col_56': 56, 'col_57': 57, 'col_58': 58, 'col_59': 59, 'col_60': 60, 'col_61': 61, 'col_62': 62, 'col_63': 63, 'col_64': 64, 'col_65': 65, 'col_66': 66, 'col_67': 67, 'col_68': 68, 'col_69': 69, 'col_70': 70, 'col_71': 71, 'col_72': 72, 'col_73': 73, 'col_74': 74, 'col_75': 75, 'col_76': 76, 'col_77': 77, 'col_78': 78, 'col_79': 79, 'col_80': 80, 'col_81': 81, 'col_82': 82, 'col_83': 83, 'col_84': 84, 'col_85': 85, 'col_86': 86, 'col_87': 87, 'col_88': 88, 'col_89': 89, 'col_90': 90, 'col_91': 91, 'col_92': 92, 'col_93': 93, 'col_94': 94, 'col_95': 95, 'col_96': 96, 'col_97': 97, 'col_98': 98, 'col_99': 99, 'rowid': 0} |
These are logs from the superset |
I don't know where to look for shillelaght's version, but I use it in superset version 3.1.0 |
You can run the following query: SELECT VERSION(); |
1.2.11 (apsw 3.42.0.1) |
@mibarannikov can you try upgrading to a newer version? |
is it possible to upgrade the shillelaght version inside the superset? Or do I need to update the entire superset? |
Can I ask another question? In the logs, first the full json is received, then some caching occurs, and the json is already cut off. Maybe it's a caching issue? |
You can upgrade only shillelagh doing |
Could be, but it could also been fixed here: https://github.com/betodealmeida/shillelagh/pull/459/files#diff-4f4bed450217073e107a02f101c43a385a304a8625fd67a86eb1cf897da922ed Let's try upgrading the version first. |
Hi. I had high hopes for the shillelagh update. Unfortunately, nothing has changed. Now I have version 1.2.26. But all field values greater than 64 are null. |
help me. I updated superset 3.1 -> 4.0.2. I updated shillagh to 1.2.27 (apsw 3.42.0.1). I created another json array source. But nothing changed. All json fields greater than 64 are null. |
Can you try upgrading apsw as well? |
This is a success! The update helped. |
Hello. All values are null in json for fields greater than 64.
The text was updated successfully, but these errors were encountered: